Command line
python /home/saxelrod/Repo/projects/chemprop/chemprop/train.py --config_path /home/saxelrod/chemprop_cov_2/models/cp3d_ndu_single_feats_no_mpnn_from_auc/config.json --data_path /home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv --dataset_type classification
Args
{'activation': 'ReLU',
 'aggregation': 'mean',
 'aggregation_norm': 100,
 'atom_descriptors': None,
 'atom_descriptors_path': None,
 'atom_descriptors_size': 0,
 'atom_features_size': 0,
 'atom_messages': False,
 'batch_size': 50,
 'bias': False,
 'cache_cutoff': 10000,
 'checkpoint_dir': None,
 'checkpoint_path': None,
 'checkpoint_paths': None,
 'class_balance': True,
 'config_path': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_single_feats_no_mpnn_from_auc/config.json',
 'crossval_index_dir': None,
 'crossval_index_file': None,
 'crossval_index_sets': None,
 'cuda': True,
 'data_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/train_full.csv',
 'dataset_type': 'classification',
 'depth': 3,
 'device': device(type='cuda', index=0),
 'dropout': 0.1,
 'ensemble_size': 1,
 'epochs': 300,
 'extra_metrics': [],
 'features_generator': None,
 'features_only': True,
 'features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_single/train_auc.npz'],
 'features_scaling': False,
 'features_size': None,
 'ffn_hidden_size': 2200,
 'ffn_num_layers': 3,
 'final_lr': 0.0001,
 'folds_file': None,
 'gpu': 0,
 'grad_clip': None,
 'hidden_size': 2200,
 'ignore_columns': None,
 'init_lr': 0.0001,
 'log_frequency': 10,
 'max_data_size': None,
 'max_lr': 0.001,
 'metric': 'auc',
 'metrics': ['auc'],
 'minimize_score': False,
 'mpn_shared': False,
 'multiclass_num_classes': 3,
 'no_cache_mol': False,
 'no_cuda': False,
 'no_features_scaling': True,
 'num_folds': 10,
 'num_lrs': 1,
 'num_tasks': 1,
 'num_workers': 8,
 'number_of_molecules': 1,
 'pytorch_seed': 0,
 'quiet': True,
 'save_dir': '/home/saxelrod/chemprop_cov_2/models/cp3d_ndu_single_feats_no_mpnn_from_auc',
 'save_preds': False,
 'save_smiles_splits': False,
 'seed': 0,
 'separate_test_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_single/test_auc.npz'],
 'separate_test_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/test_full.csv',
 'separate_val_features_path': ['/home/saxelrod/chemprop_cov_2/features/cp3d_ndu_single/val_auc.npz'],
 'separate_val_path': '/home/saxelrod/chemprop_cov_2/scaffold_split/val_full.csv',
 'show_individual_scores': False,
 'smiles_columns': [None],
 'split_sizes': (0.8, 0.1, 0.1),
 'split_type': 'random',
 'target_columns': None,
 'task_names': ['sars_cov_two_cl_protease_active'],
 'test': False,
 'test_fold_index': None,
 'train_data_size': None,
 'undirected': False,
 'use_input_features': True,
 'val_fold_index': None,
 'warmup_epochs': 2.0}
Loading data
Number of tasks = 1
Fold 0
Splitting data with seed 0
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation auc = 0.770892
Epoch 1
Validation auc = 0.766197
Epoch 2
Validation auc = 0.762911
Epoch 3
Validation auc = 0.784507
Epoch 4
Loss = 8.1724e-01, PNorm = 52.3055, GNorm = 9.8936, lr_0 = 6.5000e-04
Validation auc = 0.775587
Epoch 5
Validation auc = 0.748826
Epoch 6
Validation auc = 0.763850
Epoch 7
Validation auc = 0.753991
Epoch 8
Validation auc = 0.723474
Epoch 9
Loss = 5.8431e-01, PNorm = 52.5617, GNorm = 4.0657, lr_0 = 9.9743e-04
Validation auc = 0.761033
Epoch 10
Validation auc = 0.768075
Epoch 11
Validation auc = 0.750704
Epoch 12
Validation auc = 0.726761
Epoch 13
Validation auc = 0.732864
Epoch 14
Loss = 2.7037e-01, PNorm = 52.8628, GNorm = 1.4940, lr_0 = 9.8890e-04
Validation auc = 0.753991
Epoch 15
Validation auc = 0.756808
Epoch 16
Validation auc = 0.752113
Epoch 17
Validation auc = 0.744131
Epoch 18
Validation auc = 0.748826
Epoch 19
Loss = 1.3398e-01, PNorm = 53.1077, GNorm = 0.8356, lr_0 = 9.8045e-04
Validation auc = 0.755869
Epoch 20
Validation auc = 0.754930
Epoch 21
Validation auc = 0.756338
Epoch 22
Validation auc = 0.754460
Epoch 23
Validation auc = 0.748357
Epoch 24
Loss = 1.2135e-01, PNorm = 53.2793, GNorm = 0.5048, lr_0 = 9.7207e-04
Validation auc = 0.743192
Epoch 25
Validation auc = 0.744601
Epoch 26
Validation auc = 0.744601
Epoch 27
Validation auc = 0.745540
Epoch 28
Validation auc = 0.739437
Epoch 29
Loss = 5.0381e-02, PNorm = 53.4074, GNorm = 0.7363, lr_0 = 9.6376e-04
Validation auc = 0.738498
Epoch 30
Validation auc = 0.737559
Epoch 31
Validation auc = 0.738967
Epoch 32
Validation auc = 0.738028
Epoch 33
Validation auc = 0.732394
Epoch 34
Loss = 8.1778e-02, PNorm = 53.5153, GNorm = 1.7949, lr_0 = 9.5552e-04
Validation auc = 0.726761
Epoch 35
Validation auc = 0.728169
Epoch 36
Validation auc = 0.725352
Epoch 37
Validation auc = 0.727700
Epoch 38
Validation auc = 0.723474
Epoch 39
Loss = 7.0034e-02, PNorm = 53.6174, GNorm = 1.1204, lr_0 = 9.4735e-04
Validation auc = 0.721596
Epoch 40
Validation auc = 0.732864
Epoch 41
Validation auc = 0.732864
Epoch 42
Validation auc = 0.735681
Epoch 43
Validation auc = 0.732394
Epoch 44
Loss = 3.5295e-02, PNorm = 53.7193, GNorm = 0.2380, lr_0 = 9.3925e-04
Validation auc = 0.724413
Epoch 45
Validation auc = 0.722066
Epoch 46
Validation auc = 0.732394
Epoch 47
Validation auc = 0.740845
Epoch 48
Validation auc = 0.742723
Epoch 49
Loss = 8.3981e-02, PNorm = 53.8177, GNorm = 1.5480, lr_0 = 9.3122e-04
Validation auc = 0.736150
Epoch 50
Validation auc = 0.716901
Epoch 51
Validation auc = 0.711737
Epoch 52
Validation auc = 0.705164
Epoch 53
Validation auc = 0.705634
Epoch 54
Loss = 1.0581e-01, PNorm = 53.9204, GNorm = 0.5070, lr_0 = 9.2326e-04
Validation auc = 0.696244
Epoch 55
Validation auc = 0.698592
Epoch 56
Validation auc = 0.706103
Epoch 57
Validation auc = 0.716432
Epoch 58
Validation auc = 0.726291
Epoch 59
Loss = 8.3285e-02, PNorm = 54.0284, GNorm = 1.1655, lr_0 = 9.1537e-04
Validation auc = 0.722535
Epoch 60
Validation auc = 0.712676
Epoch 61
Validation auc = 0.711737
Epoch 62
Validation auc = 0.722535
Epoch 63
Validation auc = 0.733333
Epoch 64
Loss = 3.3011e-02, PNorm = 54.1286, GNorm = 1.3059, lr_0 = 9.0754e-04
Validation auc = 0.737559
Epoch 65
Validation auc = 0.735681
Epoch 66
Validation auc = 0.727230
Epoch 67
Validation auc = 0.717840
Epoch 68
Validation auc = 0.711737
Epoch 69
Loss = 5.8721e-02, PNorm = 54.2159, GNorm = 0.6270, lr_0 = 8.9978e-04
Validation auc = 0.706103
Epoch 70
Validation auc = 0.709390
Epoch 71
Validation auc = 0.716432
Epoch 72
Validation auc = 0.719718
Epoch 73
Validation auc = 0.723005
Epoch 74
Loss = 2.8930e-02, PNorm = 54.3084, GNorm = 0.7581, lr_0 = 8.9209e-04
Validation auc = 0.720657
Epoch 75
Validation auc = 0.720188
Epoch 76
Validation auc = 0.723474
Epoch 77
Validation auc = 0.723005
Epoch 78
Validation auc = 0.722535
Epoch 79
Loss = 2.0686e-02, PNorm = 54.3706, GNorm = 0.8687, lr_0 = 8.8447e-04
Validation auc = 0.721596
Epoch 80
Validation auc = 0.724883
Epoch 81
Validation auc = 0.725352
Epoch 82
Validation auc = 0.726761
Epoch 83
Validation auc = 0.726761
Epoch 84
Loss = 7.3882e-03, PNorm = 54.4227, GNorm = 0.1655, lr_0 = 8.7691e-04
Validation auc = 0.724883
Epoch 85
Validation auc = 0.723944
Epoch 86
Validation auc = 0.722535
Epoch 87
Validation auc = 0.723474
Epoch 88
Validation auc = 0.723005
Epoch 89
Loss = 2.9310e-02, PNorm = 54.4626, GNorm = 0.3671, lr_0 = 8.6941e-04
Validation auc = 0.721596
Epoch 90
Validation auc = 0.715023
Epoch 91
Validation auc = 0.713615
Epoch 92
Validation auc = 0.712676
Epoch 93
Validation auc = 0.713615
Epoch 94
Loss = 1.1450e-02, PNorm = 54.5105, GNorm = 0.5129, lr_0 = 8.6198e-04
Validation auc = 0.718310
Epoch 95
Validation auc = 0.722066
Epoch 96
Validation auc = 0.721127
Epoch 97
Validation auc = 0.724413
Epoch 98
Validation auc = 0.726761
Epoch 99
Loss = 1.8857e-02, PNorm = 54.5648, GNorm = 0.5059, lr_0 = 8.5461e-04
Validation auc = 0.728169
Epoch 100
Validation auc = 0.722535
Epoch 101
Validation auc = 0.719249
Epoch 102
Validation auc = 0.721127
Epoch 103
Validation auc = 0.719249
Epoch 104
Loss = 3.5516e-02, PNorm = 54.6170, GNorm = 1.1766, lr_0 = 8.4730e-04
Validation auc = 0.716901
Epoch 105
Validation auc = 0.715493
Epoch 106
Validation auc = 0.715962
Epoch 107
Validation auc = 0.715493
Epoch 108
Validation auc = 0.715493
Epoch 109
Loss = 9.2773e-03, PNorm = 54.6620, GNorm = 0.3379, lr_0 = 8.4006e-04
Validation auc = 0.713146
Epoch 110
Validation auc = 0.723944
Epoch 111
Validation auc = 0.731455
Epoch 112
Validation auc = 0.731925
Epoch 113
Validation auc = 0.740845
Epoch 114
Loss = 7.2841e-02, PNorm = 54.7221, GNorm = 0.1421, lr_0 = 8.3288e-04
Validation auc = 0.746009
Epoch 115
Validation auc = 0.750235
Epoch 116
Validation auc = 0.750235
Epoch 117
Validation auc = 0.750704
Epoch 118
Validation auc = 0.745070
Epoch 119
Loss = 1.8904e-02, PNorm = 54.7749, GNorm = 0.9282, lr_0 = 8.2576e-04
Validation auc = 0.736620
Epoch 120
Validation auc = 0.728169
Epoch 121
Validation auc = 0.725822
Epoch 122
Validation auc = 0.735211
Epoch 123
Validation auc = 0.742723
Epoch 124
Loss = 1.5066e-02, PNorm = 54.8287, GNorm = 0.0641, lr_0 = 8.1870e-04
Validation auc = 0.746009
Epoch 125
Validation auc = 0.744601
Epoch 126
Validation auc = 0.743662
Epoch 127
Validation auc = 0.734742
Epoch 128
Validation auc = 0.732394
Epoch 129
Loss = 6.4581e-02, PNorm = 54.8935, GNorm = 2.0391, lr_0 = 8.1170e-04
Validation auc = 0.733333
Epoch 130
Validation auc = 0.731455
Epoch 131
Validation auc = 0.735681
Epoch 132
Validation auc = 0.734272
Epoch 133
Validation auc = 0.736150
Epoch 134
Loss = 9.5752e-03, PNorm = 54.9571, GNorm = 0.7707, lr_0 = 8.0476e-04
Validation auc = 0.731925
Epoch 135
Validation auc = 0.730047
Epoch 136
Validation auc = 0.730047
Epoch 137
Validation auc = 0.727230
Epoch 138
Validation auc = 0.720188
Epoch 139
Loss = 5.4521e-02, PNorm = 55.0180, GNorm = 1.1977, lr_0 = 7.9788e-04
Validation auc = 0.717371
Epoch 140
Validation auc = 0.718310
Epoch 141
Validation auc = 0.716432
Epoch 142
Validation auc = 0.714554
Epoch 143
Validation auc = 0.709859
Epoch 144
Loss = 1.6617e-02, PNorm = 55.0710, GNorm = 0.6513, lr_0 = 7.9106e-04
Validation auc = 0.702347
Epoch 145
Validation auc = 0.698122
Epoch 146
Validation auc = 0.711737
Epoch 147
Validation auc = 0.719718
Epoch 148
Validation auc = 0.724413
Epoch 149
Loss = 2.3410e-03, PNorm = 55.1282, GNorm = 0.1151, lr_0 = 7.8430e-04
Validation auc = 0.726761
Epoch 150
Validation auc = 0.722535
Epoch 151
Validation auc = 0.716901
Epoch 152
Validation auc = 0.709859
Epoch 153
Validation auc = 0.711033
Epoch 154
Loss = 6.2527e-02, PNorm = 55.1637, GNorm = 0.1102, lr_0 = 7.7759e-04
Validation auc = 0.715962
Epoch 155
Validation auc = 0.715962
Epoch 156
Validation auc = 0.715962
Epoch 157
Validation auc = 0.715493
Epoch 158
Validation auc = 0.714554
Epoch 159
Loss = 8.2141e-03, PNorm = 55.1917, GNorm = 0.2397, lr_0 = 7.7095e-04
Validation auc = 0.715493
Epoch 160
Validation auc = 0.718310
Epoch 161
Validation auc = 0.721127
Epoch 162
Validation auc = 0.726291
Epoch 163
Validation auc = 0.731925
Epoch 164
Loss = 1.0429e-02, PNorm = 55.2334, GNorm = 0.5275, lr_0 = 7.6436e-04
Validation auc = 0.739906
Epoch 165
Validation auc = 0.745070
Epoch 166
Validation auc = 0.743662
Epoch 167
Validation auc = 0.738967
Epoch 168
Validation auc = 0.737559
Epoch 169
Loss = 4.9548e-03, PNorm = 55.3039, GNorm = 0.1850, lr_0 = 7.5782e-04
Validation auc = 0.736150
Epoch 170
Validation auc = 0.735681
Epoch 171
Validation auc = 0.735211
Epoch 172
Validation auc = 0.737559
Epoch 173
Validation auc = 0.737559
Epoch 174
Loss = 6.8360e-03, PNorm = 55.3531, GNorm = 0.5906, lr_0 = 7.5134e-04
Validation auc = 0.737559
Epoch 175
Validation auc = 0.736620
Epoch 176
Validation auc = 0.735681
Epoch 177
Validation auc = 0.738498
Epoch 178
Validation auc = 0.738028
Epoch 179
Loss = 4.2020e-02, PNorm = 55.3919, GNorm = 2.1283, lr_0 = 7.4492e-04
Validation auc = 0.740376
Epoch 180
Validation auc = 0.744131
Epoch 181
Validation auc = 0.745540
Epoch 182
Validation auc = 0.741784
Epoch 183
Validation auc = 0.734272
Epoch 184
Loss = 2.2349e-02, PNorm = 55.4406, GNorm = 1.2966, lr_0 = 7.3855e-04
Validation auc = 0.730047
Epoch 185
Validation auc = 0.726761
Epoch 186
Validation auc = 0.724413
Epoch 187
Validation auc = 0.719718
Epoch 188
Validation auc = 0.723005
Epoch 189
Loss = 2.0918e-02, PNorm = 55.4978, GNorm = 0.0892, lr_0 = 7.3224e-04
Validation auc = 0.724413
Epoch 190
Validation auc = 0.724883
Epoch 191
Validation auc = 0.728169
Epoch 192
Validation auc = 0.729577
Epoch 193
Validation auc = 0.731455
Epoch 194
Loss = 6.9880e-03, PNorm = 55.5495, GNorm = 0.1479, lr_0 = 7.2598e-04
Validation auc = 0.734272
Epoch 195
Validation auc = 0.734742
Epoch 196
Validation auc = 0.735681
Epoch 197
Validation auc = 0.734272
Epoch 198
Validation auc = 0.729108
Epoch 199
Loss = 1.8590e-02, PNorm = 55.5967, GNorm = 0.9816, lr_0 = 7.1977e-04
Validation auc = 0.723005
Epoch 200
Validation auc = 0.723474
Epoch 201
Validation auc = 0.720657
Epoch 202
Validation auc = 0.719249
Epoch 203
Validation auc = 0.714554
Epoch 204
Loss = 1.5938e-03, PNorm = 55.6455, GNorm = 0.0979, lr_0 = 7.1362e-04
Validation auc = 0.713146
Epoch 205
Validation auc = 0.707042
Epoch 206
Validation auc = 0.710329
Epoch 207
Validation auc = 0.712207
Epoch 208
Validation auc = 0.718779
Epoch 209
Loss = 5.3916e-02, PNorm = 55.7032, GNorm = 1.6841, lr_0 = 7.0752e-04
Validation auc = 0.724883
Epoch 210
Validation auc = 0.727230
Epoch 211
Validation auc = 0.727700
Epoch 212
Validation auc = 0.725822
Epoch 213
Validation auc = 0.724883
Epoch 214
Loss = 1.1785e-02, PNorm = 55.7801, GNorm = 0.5773, lr_0 = 7.0147e-04
Validation auc = 0.727230
Epoch 215
Validation auc = 0.724883
Epoch 216
Validation auc = 0.719249
Epoch 217
Validation auc = 0.716432
Epoch 218
Validation auc = 0.712676
Epoch 219
Loss = 3.1909e-02, PNorm = 55.8546, GNorm = 1.9518, lr_0 = 6.9548e-04
Validation auc = 0.715962
Epoch 220
Validation auc = 0.719718
Epoch 221
Validation auc = 0.722535
Epoch 222
Validation auc = 0.726291
Epoch 223
Validation auc = 0.727230
Epoch 224
Loss = 3.2640e-02, PNorm = 55.9184, GNorm = 1.8898, lr_0 = 6.8953e-04
Validation auc = 0.725822
Epoch 225
Validation auc = 0.726291
Epoch 226
Validation auc = 0.723944
Epoch 227
Validation auc = 0.724883
Epoch 228
Validation auc = 0.725352
Epoch 229
Loss = 5.5227e-03, PNorm = 55.9839, GNorm = 0.1371, lr_0 = 6.8364e-04
Validation auc = 0.728638
Epoch 230
Validation auc = 0.733803
Epoch 231
Validation auc = 0.735681
Epoch 232
Validation auc = 0.739906
Epoch 233
Validation auc = 0.747887
Epoch 234
Loss = 2.6166e-02, PNorm = 56.0322, GNorm = 1.4928, lr_0 = 6.7779e-04
Validation auc = 0.745540
Epoch 235
Validation auc = 0.740376
Epoch 236
Validation auc = 0.735681
Epoch 237
Validation auc = 0.729108
Epoch 238
Validation auc = 0.724413
Epoch 239
Loss = 5.4588e-04, PNorm = 56.0713, GNorm = 0.0171, lr_0 = 6.7200e-04
Validation auc = 0.721127
Epoch 240
Validation auc = 0.722535
Epoch 241
Validation auc = 0.723944
Epoch 242
Validation auc = 0.724413
Epoch 243
Validation auc = 0.725822
Epoch 244
Loss = 2.3171e-02, PNorm = 56.1013, GNorm = 1.3649, lr_0 = 6.6625e-04
Validation auc = 0.723474
Epoch 245
Validation auc = 0.722066
Epoch 246
Validation auc = 0.722535
Epoch 247
Validation auc = 0.723005
Epoch 248
Validation auc = 0.724883
Epoch 249
Loss = 6.8908e-03, PNorm = 56.1344, GNorm = 0.2922, lr_0 = 6.6056e-04
Validation auc = 0.726761
Epoch 250
Validation auc = 0.729577
Epoch 251
Validation auc = 0.733333
Epoch 252
Validation auc = 0.731455
Epoch 253
Validation auc = 0.729577
Epoch 254
Loss = 3.4813e-03, PNorm = 56.1587, GNorm = 0.2489, lr_0 = 6.5491e-04
Validation auc = 0.730516
Epoch 255
Validation auc = 0.730516
Epoch 256
Validation auc = 0.728169
Epoch 257
Validation auc = 0.726761
Epoch 258
Validation auc = 0.727230
Epoch 259
Loss = 1.5124e-02, PNorm = 56.1805, GNorm = 0.1980, lr_0 = 6.4931e-04
Validation auc = 0.728638
Epoch 260
Validation auc = 0.729108
Epoch 261
Validation auc = 0.729577
Epoch 262
Validation auc = 0.730986
Epoch 263
Validation auc = 0.726291
Epoch 264
Loss = 2.6610e-03, PNorm = 56.2010, GNorm = 0.1616, lr_0 = 6.4376e-04
Validation auc = 0.723005
Epoch 265
Validation auc = 0.719718
Epoch 266
Validation auc = 0.720188
Epoch 267
Validation auc = 0.718310
Epoch 268
Validation auc = 0.724413
Epoch 269
Loss = 4.2686e-03, PNorm = 56.2288, GNorm = 0.0806, lr_0 = 6.3826e-04
Validation auc = 0.731690
Epoch 270
Validation auc = 0.736620
Epoch 271
Validation auc = 0.739437
Epoch 272
Validation auc = 0.744131
Epoch 273
Validation auc = 0.744601
Epoch 274
Loss = 7.9753e-03, PNorm = 56.2552, GNorm = 0.5523, lr_0 = 6.3280e-04
Validation auc = 0.743192
Epoch 275
Validation auc = 0.741315
Epoch 276
Validation auc = 0.738967
Epoch 277
Validation auc = 0.739906
Epoch 278
Validation auc = 0.737324
Epoch 279
Loss = 1.4868e-03, PNorm = 56.2799, GNorm = 0.2029, lr_0 = 6.2739e-04
Validation auc = 0.736385
Epoch 280
Validation auc = 0.735211
Epoch 281
Validation auc = 0.735211
Epoch 282
Validation auc = 0.735681
Epoch 283
Validation auc = 0.738967
Epoch 284
Loss = 8.9717e-03, PNorm = 56.2986, GNorm = 0.8325, lr_0 = 6.2203e-04
Validation auc = 0.741315
Epoch 285
Validation auc = 0.741315
Epoch 286
Validation auc = 0.743192
Epoch 287
Validation auc = 0.741784
Epoch 288
Validation auc = 0.742254
Epoch 289
Loss = 2.8535e-04, PNorm = 56.3168, GNorm = 0.0174, lr_0 = 6.1671e-04
Validation auc = 0.741080
Epoch 290
Validation auc = 0.741315
Epoch 291
Validation auc = 0.741549
Epoch 292
Validation auc = 0.740376
Epoch 293
Validation auc = 0.739906
Epoch 294
Loss = 7.1480e-04, PNorm = 56.3336, GNorm = 0.0018, lr_0 = 6.1144e-04
Validation auc = 0.739437
Epoch 295
Validation auc = 0.740376
Epoch 296
Validation auc = 0.740376
Epoch 297
Validation auc = 0.740376
Epoch 298
Validation auc = 0.739437
Epoch 299
Loss = 8.5129e-04, PNorm = 56.3533, GNorm = 0.0618, lr_0 = 6.0621e-04
Validation auc = 0.739437
Model 0 best validation auc = 0.784507 on epoch 3
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.941601
Ensemble test auc = 0.941601
Fold 1
Splitting data with seed 1
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation auc = 0.750704
Epoch 1
Validation auc = 0.750704
Epoch 2
Validation auc = 0.769014
Epoch 3
Validation auc = 0.761972
Epoch 4
Loss = 4.6267e-01, PNorm = 52.3000, GNorm = 6.0041, lr_0 = 6.5000e-04
Validation auc = 0.747887
Epoch 5
Validation auc = 0.736620
Epoch 6
Validation auc = 0.744131
Epoch 7
Validation auc = 0.750235
Epoch 8
Validation auc = 0.730516
Epoch 9
Loss = 3.9666e-01, PNorm = 52.5506, GNorm = 5.9391, lr_0 = 9.9743e-04
Validation auc = 0.736150
Epoch 10
Validation auc = 0.709859
Epoch 11
Validation auc = 0.662911
Epoch 12
Validation auc = 0.719718
Epoch 13
Validation auc = 0.732864
Epoch 14
Loss = 2.6597e-01, PNorm = 52.8374, GNorm = 1.1178, lr_0 = 9.8890e-04
Validation auc = 0.722066
Epoch 15
Validation auc = 0.734272
Epoch 16
Validation auc = 0.757746
Epoch 17
Validation auc = 0.759624
Epoch 18
Validation auc = 0.754930
Epoch 19
Loss = 1.1557e-01, PNorm = 53.0763, GNorm = 0.8146, lr_0 = 9.8045e-04
Validation auc = 0.735211
Epoch 20
Validation auc = 0.736150
Epoch 21
Validation auc = 0.735211
Epoch 22
Validation auc = 0.735681
Epoch 23
Validation auc = 0.730516
Epoch 24
Loss = 9.2042e-02, PNorm = 53.2512, GNorm = 1.2950, lr_0 = 9.7207e-04
Validation auc = 0.723944
Epoch 25
Validation auc = 0.726761
Epoch 26
Validation auc = 0.738498
Epoch 27
Validation auc = 0.746009
Epoch 28
Validation auc = 0.747418
Epoch 29
Loss = 8.3336e-02, PNorm = 53.3765, GNorm = 1.0509, lr_0 = 9.6376e-04
Validation auc = 0.745540
Epoch 30
Validation auc = 0.751643
Epoch 31
Validation auc = 0.746479
Epoch 32
Validation auc = 0.742254
Epoch 33
Validation auc = 0.731455
Epoch 34
Loss = 4.1803e-02, PNorm = 53.4873, GNorm = 0.3144, lr_0 = 9.5552e-04
Validation auc = 0.723474
Epoch 35
Validation auc = 0.723944
Epoch 36
Validation auc = 0.736150
Epoch 37
Validation auc = 0.747418
Epoch 38
Validation auc = 0.749296
Epoch 39
Loss = 1.1407e-01, PNorm = 53.5872, GNorm = 1.4928, lr_0 = 9.4735e-04
Validation auc = 0.744131
Epoch 40
Validation auc = 0.738967
Epoch 41
Validation auc = 0.737559
Epoch 42
Validation auc = 0.732864
Epoch 43
Validation auc = 0.724883
Epoch 44
Loss = 6.2422e-02, PNorm = 53.6735, GNorm = 1.1367, lr_0 = 9.3925e-04
Validation auc = 0.721596
Epoch 45
Validation auc = 0.729577
Epoch 46
Validation auc = 0.737089
Epoch 47
Validation auc = 0.730047
Epoch 48
Validation auc = 0.730516
Epoch 49
Loss = 3.2015e-02, PNorm = 53.7570, GNorm = 0.4897, lr_0 = 9.3122e-04
Validation auc = 0.724883
Epoch 50
Validation auc = 0.707512
Epoch 51
Validation auc = 0.694366
Epoch 52
Validation auc = 0.688732
Epoch 53
Validation auc = 0.688263
Epoch 54
Loss = 7.8423e-02, PNorm = 53.8358, GNorm = 1.6566, lr_0 = 9.2326e-04
Validation auc = 0.708920
Epoch 55
Validation auc = 0.723944
Epoch 56
Validation auc = 0.715023
Epoch 57
Validation auc = 0.715493
Epoch 58
Validation auc = 0.730516
Epoch 59
Loss = 7.7378e-02, PNorm = 53.9345, GNorm = 1.7860, lr_0 = 9.1537e-04
Validation auc = 0.738967
Epoch 60
Validation auc = 0.746009
Epoch 61
Validation auc = 0.745070
Epoch 62
Validation auc = 0.721596
Epoch 63
Validation auc = 0.708920
Epoch 64
Loss = 2.2585e-01, PNorm = 54.0462, GNorm = 1.3305, lr_0 = 9.0754e-04
Validation auc = 0.732394
Epoch 65
Validation auc = 0.728638
Epoch 66
Validation auc = 0.725352
Epoch 67
Validation auc = 0.721596
Epoch 68
Validation auc = 0.731925
Epoch 69
Loss = 1.1213e-01, PNorm = 54.2320, GNorm = 1.2599, lr_0 = 8.9978e-04
Validation auc = 0.746948
Epoch 70
Validation auc = 0.750235
Epoch 71
Validation auc = 0.740845
Epoch 72
Validation auc = 0.729108
Epoch 73
Validation auc = 0.716432
Epoch 74
Loss = 8.6442e-02, PNorm = 54.3816, GNorm = 0.2163, lr_0 = 8.9209e-04
Validation auc = 0.717840
Epoch 75
Validation auc = 0.725352
Epoch 76
Validation auc = 0.730986
Epoch 77
Validation auc = 0.734742
Epoch 78
Validation auc = 0.730986
Epoch 79
Loss = 4.6327e-02, PNorm = 54.5051, GNorm = 0.0943, lr_0 = 8.8447e-04
Validation auc = 0.722066
Epoch 80
Validation auc = 0.716901
Epoch 81
Validation auc = 0.722535
Epoch 82
Validation auc = 0.723474
Epoch 83
Validation auc = 0.724883
Epoch 84
Loss = 1.8561e-02, PNorm = 54.5881, GNorm = 0.3893, lr_0 = 8.7691e-04
Validation auc = 0.723005
Epoch 85
Validation auc = 0.722300
Epoch 86
Validation auc = 0.723944
Epoch 87
Validation auc = 0.729108
Epoch 88
Validation auc = 0.744131
Epoch 89
Loss = 8.8538e-03, PNorm = 54.6574, GNorm = 0.2456, lr_0 = 8.6941e-04
Validation auc = 0.752582
Epoch 90
Validation auc = 0.750235
Epoch 91
Validation auc = 0.747887
Epoch 92
Validation auc = 0.744601
Epoch 93
Validation auc = 0.746948
Epoch 94
Loss = 3.2106e-02, PNorm = 54.7357, GNorm = 0.7609, lr_0 = 8.6198e-04
Validation auc = 0.755399
Epoch 95
Validation auc = 0.752582
Epoch 96
Validation auc = 0.749765
Epoch 97
Validation auc = 0.745540
Epoch 98
Validation auc = 0.744601
Epoch 99
Loss = 1.5509e-02, PNorm = 54.8095, GNorm = 0.4280, lr_0 = 8.5461e-04
Validation auc = 0.738498
Epoch 100
Validation auc = 0.733803
Epoch 101
Validation auc = 0.723005
Epoch 102
Validation auc = 0.720188
Epoch 103
Validation auc = 0.717371
Epoch 104
Loss = 9.8105e-03, PNorm = 54.8743, GNorm = 0.0508, lr_0 = 8.4730e-04
Validation auc = 0.715493
Epoch 105
Validation auc = 0.714085
Epoch 106
Validation auc = 0.713615
Epoch 107
Validation auc = 0.714085
Epoch 108
Validation auc = 0.717371
Epoch 109
Loss = 5.3898e-02, PNorm = 54.9335, GNorm = 2.4248, lr_0 = 8.4006e-04
Validation auc = 0.730047
Epoch 110
Validation auc = 0.738028
Epoch 111
Validation auc = 0.744131
Epoch 112
Validation auc = 0.748826
Epoch 113
Validation auc = 0.756808
Epoch 114
Loss = 1.5406e-02, PNorm = 54.9983, GNorm = 0.0358, lr_0 = 8.3288e-04
Validation auc = 0.754460
Epoch 115
Validation auc = 0.750704
Epoch 116
Validation auc = 0.752347
Epoch 117
Validation auc = 0.752582
Epoch 118
Validation auc = 0.747418
Epoch 119
Loss = 4.5096e-02, PNorm = 55.0714, GNorm = 2.1917, lr_0 = 8.2576e-04
Validation auc = 0.749296
Epoch 120
Validation auc = 0.732394
Epoch 121
Validation auc = 0.711737
Epoch 122
Validation auc = 0.698826
Epoch 123
Validation auc = 0.695775
Epoch 124
Loss = 5.6548e-02, PNorm = 55.1937, GNorm = 0.9029, lr_0 = 8.1870e-04
Validation auc = 0.707512
Epoch 125
Validation auc = 0.721127
Epoch 126
Validation auc = 0.726291
Epoch 127
Validation auc = 0.717371
Epoch 128
Validation auc = 0.707981
Epoch 129
Loss = 4.7093e-02, PNorm = 55.3078, GNorm = 1.2875, lr_0 = 8.1170e-04
Validation auc = 0.700939
Epoch 130
Validation auc = 0.696244
Epoch 131
Validation auc = 0.700000
Epoch 132
Validation auc = 0.705634
Epoch 133
Validation auc = 0.705634
Epoch 134
Loss = 4.1726e-02, PNorm = 55.4197, GNorm = 0.8864, lr_0 = 8.0476e-04
Validation auc = 0.704695
Epoch 135
Validation auc = 0.702347
Epoch 136
Validation auc = 0.695305
Epoch 137
Validation auc = 0.696714
Epoch 138
Validation auc = 0.697653
Epoch 139
Loss = 5.7840e-03, PNorm = 55.5013, GNorm = 0.0983, lr_0 = 7.9788e-04
Validation auc = 0.701408
Epoch 140
Validation auc = 0.706573
Epoch 141
Validation auc = 0.709859
Epoch 142
Validation auc = 0.711737
Epoch 143
Validation auc = 0.715023
Epoch 144
Loss = 1.2432e-02, PNorm = 55.5501, GNorm = 0.1635, lr_0 = 7.9106e-04
Validation auc = 0.712207
Epoch 145
Validation auc = 0.714085
Epoch 146
Validation auc = 0.715962
Epoch 147
Validation auc = 0.715962
Epoch 148
Validation auc = 0.715493
Epoch 149
Loss = 3.8930e-02, PNorm = 55.5846, GNorm = 1.1580, lr_0 = 7.8430e-04
Validation auc = 0.716901
Epoch 150
Validation auc = 0.718779
Epoch 151
Validation auc = 0.719718
Epoch 152
Validation auc = 0.720188
Epoch 153
Validation auc = 0.722066
Epoch 154
Loss = 1.2217e-02, PNorm = 55.6306, GNorm = 0.4368, lr_0 = 7.7759e-04
Validation auc = 0.721596
Epoch 155
Validation auc = 0.725352
Epoch 156
Validation auc = 0.725587
Epoch 157
Validation auc = 0.724883
Epoch 158
Validation auc = 0.725587
Epoch 159
Loss = 1.5204e-03, PNorm = 55.6703, GNorm = 0.0504, lr_0 = 7.7095e-04
Validation auc = 0.725822
Epoch 160
Validation auc = 0.726291
Epoch 161
Validation auc = 0.724883
Epoch 162
Validation auc = 0.715023
Epoch 163
Validation auc = 0.714085
Epoch 164
Loss = 1.0039e-02, PNorm = 55.6964, GNorm = 0.6617, lr_0 = 7.6436e-04
Validation auc = 0.711737
Epoch 165
Validation auc = 0.707512
Epoch 166
Validation auc = 0.701878
Epoch 167
Validation auc = 0.698122
Epoch 168
Validation auc = 0.698122
Epoch 169
Loss = 7.9620e-03, PNorm = 55.7307, GNorm = 0.0690, lr_0 = 7.5782e-04
Validation auc = 0.704225
Epoch 170
Validation auc = 0.710329
Epoch 171
Validation auc = 0.710798
Epoch 172
Validation auc = 0.712207
Epoch 173
Validation auc = 0.714554
Epoch 174
Loss = 1.0091e-02, PNorm = 55.7675, GNorm = 0.8911, lr_0 = 7.5134e-04
Validation auc = 0.714319
Epoch 175
Validation auc = 0.713850
Epoch 176
Validation auc = 0.714085
Epoch 177
Validation auc = 0.713615
Epoch 178
Validation auc = 0.711502
Epoch 179
Loss = 2.0720e-02, PNorm = 55.7951, GNorm = 0.9435, lr_0 = 7.4492e-04
Validation auc = 0.715728
Epoch 180
Validation auc = 0.718545
Epoch 181
Validation auc = 0.722066
Epoch 182
Validation auc = 0.723005
Epoch 183
Validation auc = 0.722535
Epoch 184
Loss = 1.0167e-02, PNorm = 55.8214, GNorm = 0.8273, lr_0 = 7.3855e-04
Validation auc = 0.720188
Epoch 185
Validation auc = 0.721596
Epoch 186
Validation auc = 0.720657
Epoch 187
Validation auc = 0.718310
Epoch 188
Validation auc = 0.714085
Epoch 189
Loss = 8.3008e-03, PNorm = 55.8581, GNorm = 0.4731, lr_0 = 7.3224e-04
Validation auc = 0.709859
Epoch 190
Validation auc = 0.707746
Epoch 191
Validation auc = 0.707981
Epoch 192
Validation auc = 0.710563
Epoch 193
Validation auc = 0.712911
Epoch 194
Loss = 3.1478e-02, PNorm = 55.8961, GNorm = 1.6203, lr_0 = 7.2598e-04
Validation auc = 0.712676
Epoch 195
Validation auc = 0.718075
Epoch 196
Validation auc = 0.720188
Epoch 197
Validation auc = 0.724178
Epoch 198
Validation auc = 0.726056
Epoch 199
Loss = 1.0732e-02, PNorm = 55.9325, GNorm = 0.0270, lr_0 = 7.1977e-04
Validation auc = 0.725117
Epoch 200
Validation auc = 0.725117
Epoch 201
Validation auc = 0.724648
Epoch 202
Validation auc = 0.719484
Epoch 203
Validation auc = 0.719484
Epoch 204
Loss = 1.5015e-02, PNorm = 55.9530, GNorm = 0.0454, lr_0 = 7.1362e-04
Validation auc = 0.716667
Epoch 205
Validation auc = 0.717371
Epoch 206
Validation auc = 0.711737
Epoch 207
Validation auc = 0.708685
Epoch 208
Validation auc = 0.706808
Epoch 209
Loss = 1.0327e-02, PNorm = 55.9855, GNorm = 0.6900, lr_0 = 7.0752e-04
Validation auc = 0.703991
Epoch 210
Validation auc = 0.709155
Epoch 211
Validation auc = 0.719484
Epoch 212
Validation auc = 0.730047
Epoch 213
Validation auc = 0.734272
Epoch 214
Loss = 1.7870e-02, PNorm = 56.0386, GNorm = 1.2739, lr_0 = 7.0147e-04
Validation auc = 0.738028
Epoch 215
Validation auc = 0.738967
Epoch 216
Validation auc = 0.730986
Epoch 217
Validation auc = 0.730047
Epoch 218
Validation auc = 0.727230
Epoch 219
Loss = 1.0880e-02, PNorm = 56.0856, GNorm = 0.7826, lr_0 = 6.9548e-04
Validation auc = 0.723944
Epoch 220
Validation auc = 0.723944
Epoch 221
Validation auc = 0.724413
Epoch 222
Validation auc = 0.727230
Epoch 223
Validation auc = 0.730047
Epoch 224
Loss = 5.2029e-03, PNorm = 56.1402, GNorm = 0.6995, lr_0 = 6.8953e-04
Validation auc = 0.730986
Epoch 225
Validation auc = 0.730516
Epoch 226
Validation auc = 0.733803
Epoch 227
Validation auc = 0.730986
Epoch 228
Validation auc = 0.727700
Epoch 229
Loss = 1.3522e-03, PNorm = 56.1773, GNorm = 0.1170, lr_0 = 6.8364e-04
Validation auc = 0.727700
Epoch 230
Validation auc = 0.721596
Epoch 231
Validation auc = 0.718310
Epoch 232
Validation auc = 0.722535
Epoch 233
Validation auc = 0.725352
Epoch 234
Loss = 6.1745e-03, PNorm = 56.2111, GNorm = 0.6623, lr_0 = 6.7779e-04
Validation auc = 0.728638
Epoch 235
Validation auc = 0.732864
Epoch 236
Validation auc = 0.734272
Epoch 237
Validation auc = 0.735681
Epoch 238
Validation auc = 0.736620
Epoch 239
Loss = 1.9308e-03, PNorm = 56.2371, GNorm = 0.1388, lr_0 = 6.7200e-04
Validation auc = 0.738967
Epoch 240
Validation auc = 0.738498
Epoch 241
Validation auc = 0.734742
Epoch 242
Validation auc = 0.733333
Epoch 243
Validation auc = 0.730516
Epoch 244
Loss = 3.8334e-02, PNorm = 56.2603, GNorm = 2.1944, lr_0 = 6.6625e-04
Validation auc = 0.728638
Epoch 245
Validation auc = 0.730047
Epoch 246
Validation auc = 0.730516
Epoch 247
Validation auc = 0.730516
Epoch 248
Validation auc = 0.729108
Epoch 249
Loss = 7.2306e-03, PNorm = 56.2847, GNorm = 1.0336, lr_0 = 6.6056e-04
Validation auc = 0.725352
Epoch 250
Validation auc = 0.727230
Epoch 251
Validation auc = 0.723474
Epoch 252
Validation auc = 0.723005
Epoch 253
Validation auc = 0.722066
Epoch 254
Loss = 2.5491e-03, PNorm = 56.3157, GNorm = 0.1834, lr_0 = 6.5491e-04
Validation auc = 0.721127
Epoch 255
Validation auc = 0.722066
Epoch 256
Validation auc = 0.723005
Epoch 257
Validation auc = 0.722535
Epoch 258
Validation auc = 0.721127
Epoch 259
Loss = 2.7050e-03, PNorm = 56.3355, GNorm = 0.1449, lr_0 = 6.4931e-04
Validation auc = 0.721127
Epoch 260
Validation auc = 0.720188
Epoch 261
Validation auc = 0.719718
Epoch 262
Validation auc = 0.718779
Epoch 263
Validation auc = 0.719718
Epoch 264
Loss = 6.0335e-04, PNorm = 56.3493, GNorm = 0.0226, lr_0 = 6.4376e-04
Validation auc = 0.720188
Epoch 265
Validation auc = 0.720657
Epoch 266
Validation auc = 0.719249
Epoch 267
Validation auc = 0.717371
Epoch 268
Validation auc = 0.715493
Epoch 269
Loss = 9.9058e-04, PNorm = 56.3625, GNorm = 0.1285, lr_0 = 6.3826e-04
Validation auc = 0.714554
Epoch 270
Validation auc = 0.712207
Epoch 271
Validation auc = 0.712207
Epoch 272
Validation auc = 0.715493
Epoch 273
Validation auc = 0.719718
Epoch 274
Loss = 8.3230e-04, PNorm = 56.3777, GNorm = 0.0197, lr_0 = 6.3280e-04
Validation auc = 0.724883
Epoch 275
Validation auc = 0.724413
Epoch 276
Validation auc = 0.724413
Epoch 277
Validation auc = 0.723474
Epoch 278
Validation auc = 0.717371
Epoch 279
Loss = 6.6390e-02, PNorm = 56.3921, GNorm = 0.0764, lr_0 = 6.2739e-04
Validation auc = 0.713615
Epoch 280
Validation auc = 0.715493
Epoch 281
Validation auc = 0.718779
Epoch 282
Validation auc = 0.721596
Epoch 283
Validation auc = 0.722535
Epoch 284
Loss = 6.1766e-03, PNorm = 56.4118, GNorm = 0.5205, lr_0 = 6.2203e-04
Validation auc = 0.721596
Epoch 285
Validation auc = 0.720188
Epoch 286
Validation auc = 0.720657
Epoch 287
Validation auc = 0.719249
Epoch 288
Validation auc = 0.717840
Epoch 289
Loss = 2.2875e-03, PNorm = 56.4296, GNorm = 0.1396, lr_0 = 6.1671e-04
Validation auc = 0.715962
Epoch 290
Validation auc = 0.714554
Epoch 291
Validation auc = 0.713615
Epoch 292
Validation auc = 0.718779
Epoch 293
Validation auc = 0.721596
Epoch 294
Loss = 3.1518e-02, PNorm = 56.4515, GNorm = 0.7631, lr_0 = 6.1144e-04
Validation auc = 0.720657
Epoch 295
Validation auc = 0.724883
Epoch 296
Validation auc = 0.722535
Epoch 297
Validation auc = 0.724883
Epoch 298
Validation auc = 0.724883
Epoch 299
Loss = 1.7215e-02, PNorm = 56.4844, GNorm = 1.1901, lr_0 = 6.0621e-04
Validation auc = 0.729577
Model 0 best validation auc = 0.769014 on epoch 2
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.886213
Ensemble test auc = 0.886213
Fold 2
Splitting data with seed 2
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation auc = 0.741315
Epoch 1
Validation auc = 0.740376
Epoch 2
Validation auc = 0.759624
Epoch 3
Validation auc = 0.762911
Epoch 4
Loss = 4.9878e-01, PNorm = 52.3021, GNorm = 5.0751, lr_0 = 6.5000e-04
Validation auc = 0.745540
Epoch 5
Validation auc = 0.733803
Epoch 6
Validation auc = 0.751643
Epoch 7
Validation auc = 0.743662
Epoch 8
Validation auc = 0.766197
Epoch 9
Loss = 4.9858e-01, PNorm = 52.5517, GNorm = 2.9559, lr_0 = 9.9743e-04
Validation auc = 0.761033
Epoch 10
Validation auc = 0.752113
Epoch 11
Validation auc = 0.746948
Epoch 12
Validation auc = 0.732394
Epoch 13
Validation auc = 0.730516
Epoch 14
Loss = 1.7406e-01, PNorm = 52.8358, GNorm = 2.5568, lr_0 = 9.8890e-04
Validation auc = 0.739437
Epoch 15
Validation auc = 0.734742
Epoch 16
Validation auc = 0.738967
Epoch 17
Validation auc = 0.743662
Epoch 18
Validation auc = 0.735681
Epoch 19
Loss = 1.1129e-01, PNorm = 53.0429, GNorm = 0.5822, lr_0 = 9.8045e-04
Validation auc = 0.713146
Epoch 20
Validation auc = 0.725822
Epoch 21
Validation auc = 0.749765
Epoch 22
Validation auc = 0.743662
Epoch 23
Validation auc = 0.739906
Epoch 24
Loss = 7.7886e-02, PNorm = 53.2191, GNorm = 0.9135, lr_0 = 9.7207e-04
Validation auc = 0.733803
Epoch 25
Validation auc = 0.740376
Epoch 26
Validation auc = 0.744601
Epoch 27
Validation auc = 0.746948
Epoch 28
Validation auc = 0.748826
Epoch 29
Loss = 1.5812e-01, PNorm = 53.3619, GNorm = 1.1443, lr_0 = 9.6376e-04
Validation auc = 0.742254
Epoch 30
Validation auc = 0.734742
Epoch 31
Validation auc = 0.719249
Epoch 32
Validation auc = 0.700469
Epoch 33
Validation auc = 0.705164
Epoch 34
Loss = 1.2104e-01, PNorm = 53.4963, GNorm = 2.1022, lr_0 = 9.5552e-04
Validation auc = 0.708920
Epoch 35
Validation auc = 0.715023
Epoch 36
Validation auc = 0.743192
Epoch 37
Validation auc = 0.763850
Epoch 38
Validation auc = 0.767606
Epoch 39
Loss = 8.8274e-02, PNorm = 53.6217, GNorm = 1.1886, lr_0 = 9.4735e-04
Validation auc = 0.757746
Epoch 40
Validation auc = 0.757277
Epoch 41
Validation auc = 0.759624
Epoch 42
Validation auc = 0.760094
Epoch 43
Validation auc = 0.760563
Epoch 44
Loss = 7.9743e-02, PNorm = 53.7448, GNorm = 0.7515, lr_0 = 9.3925e-04
Validation auc = 0.759624
Epoch 45
Validation auc = 0.754460
Epoch 46
Validation auc = 0.748357
Epoch 47
Validation auc = 0.741315
Epoch 48
Validation auc = 0.737559
Epoch 49
Loss = 5.7130e-02, PNorm = 53.8493, GNorm = 0.7167, lr_0 = 9.3122e-04
Validation auc = 0.733333
Epoch 50
Validation auc = 0.727700
Epoch 51
Validation auc = 0.725822
Epoch 52
Validation auc = 0.731455
Epoch 53
Validation auc = 0.735681
Epoch 54
Loss = 5.6191e-02, PNorm = 53.9401, GNorm = 1.0969, lr_0 = 9.2326e-04
Validation auc = 0.745540
Epoch 55
Validation auc = 0.746479
Epoch 56
Validation auc = 0.743662
Epoch 57
Validation auc = 0.739906
Epoch 58
Validation auc = 0.738498
Epoch 59
Loss = 5.6884e-02, PNorm = 54.0389, GNorm = 2.0390, lr_0 = 9.1537e-04
Validation auc = 0.734272
Epoch 60
Validation auc = 0.737559
Epoch 61
Validation auc = 0.742254
Epoch 62
Validation auc = 0.753521
Epoch 63
Validation auc = 0.753991
Epoch 64
Loss = 1.9120e-02, PNorm = 54.1584, GNorm = 0.4355, lr_0 = 9.0754e-04
Validation auc = 0.754930
Epoch 65
Validation auc = 0.748826
Epoch 66
Validation auc = 0.746479
Epoch 67
Validation auc = 0.742723
Epoch 68
Validation auc = 0.738498
Epoch 69
Loss = 1.4227e-02, PNorm = 54.2475, GNorm = 0.5720, lr_0 = 8.9978e-04
Validation auc = 0.731925
Epoch 70
Validation auc = 0.726291
Epoch 71
Validation auc = 0.723944
Epoch 72
Validation auc = 0.726291
Epoch 73
Validation auc = 0.728638
Epoch 74
Loss = 8.2973e-03, PNorm = 54.3222, GNorm = 0.1554, lr_0 = 8.9209e-04
Validation auc = 0.732394
Epoch 75
Validation auc = 0.734272
Epoch 76
Validation auc = 0.742723
Epoch 77
Validation auc = 0.744601
Epoch 78
Validation auc = 0.732394
Epoch 79
Loss = 7.1592e-02, PNorm = 54.3923, GNorm = 1.9628, lr_0 = 8.8447e-04
Validation auc = 0.719249
Epoch 80
Validation auc = 0.713615
Epoch 81
Validation auc = 0.716432
Epoch 82
Validation auc = 0.728169
Epoch 83
Validation auc = 0.731455
Epoch 84
Loss = 5.5902e-02, PNorm = 54.4919, GNorm = 1.5121, lr_0 = 8.7691e-04
Validation auc = 0.732864
Epoch 85
Validation auc = 0.737089
Epoch 86
Validation auc = 0.730986
Epoch 87
Validation auc = 0.724883
Epoch 88
Validation auc = 0.726291
Epoch 89
Loss = 2.2015e-02, PNorm = 54.5751, GNorm = 0.7044, lr_0 = 8.6941e-04
Validation auc = 0.733803
Epoch 90
Validation auc = 0.738498
Epoch 91
Validation auc = 0.742723
Epoch 92
Validation auc = 0.741315
Epoch 93
Validation auc = 0.740845
Epoch 94
Loss = 7.9590e-03, PNorm = 54.6498, GNorm = 0.2287, lr_0 = 8.6198e-04
Validation auc = 0.741315
Epoch 95
Validation auc = 0.740845
Epoch 96
Validation auc = 0.741784
Epoch 97
Validation auc = 0.740376
Epoch 98
Validation auc = 0.740845
Epoch 99
Loss = 2.4213e-02, PNorm = 54.6989, GNorm = 0.5465, lr_0 = 8.5461e-04
Validation auc = 0.739437
Epoch 100
Validation auc = 0.736150
Epoch 101
Validation auc = 0.730516
Epoch 102
Validation auc = 0.732394
Epoch 103
Validation auc = 0.733803
Epoch 104
Loss = 5.4476e-02, PNorm = 54.7544, GNorm = 0.3125, lr_0 = 8.4730e-04
Validation auc = 0.731455
Epoch 105
Validation auc = 0.725352
Epoch 106
Validation auc = 0.720657
Epoch 107
Validation auc = 0.713615
Epoch 108
Validation auc = 0.711268
Epoch 109
Loss = 1.3892e-01, PNorm = 54.8101, GNorm = 1.8452, lr_0 = 8.4006e-04
Validation auc = 0.721596
Epoch 110
Validation auc = 0.731455
Epoch 111
Validation auc = 0.731925
Epoch 112
Validation auc = 0.730986
Epoch 113
Validation auc = 0.730047
Epoch 114
Loss = 2.1649e-02, PNorm = 54.8688, GNorm = 0.8436, lr_0 = 8.3288e-04
Validation auc = 0.716901
Epoch 115
Validation auc = 0.700000
Epoch 116
Validation auc = 0.701878
Epoch 117
Validation auc = 0.699531
Epoch 118
Validation auc = 0.709390
Epoch 119
Loss = 2.7729e-02, PNorm = 54.9432, GNorm = 0.7996, lr_0 = 8.2576e-04
Validation auc = 0.714085
Epoch 120
Validation auc = 0.720188
Epoch 121
Validation auc = 0.725822
Epoch 122
Validation auc = 0.726761
Epoch 123
Validation auc = 0.733333
Epoch 124
Loss = 3.2753e-02, PNorm = 55.0194, GNorm = 1.3875, lr_0 = 8.1870e-04
Validation auc = 0.731455
Epoch 125
Validation auc = 0.727230
Epoch 126
Validation auc = 0.723944
Epoch 127
Validation auc = 0.721596
Epoch 128
Validation auc = 0.721127
Epoch 129
Loss = 6.3999e-03, PNorm = 55.0906, GNorm = 0.4817, lr_0 = 8.1170e-04
Validation auc = 0.719718
Epoch 130
Validation auc = 0.718779
Epoch 131
Validation auc = 0.717371
Epoch 132
Validation auc = 0.720188
Epoch 133
Validation auc = 0.721596
Epoch 134
Loss = 8.6305e-03, PNorm = 55.1394, GNorm = 0.7466, lr_0 = 8.0476e-04
Validation auc = 0.722535
Epoch 135
Validation auc = 0.721127
Epoch 136
Validation auc = 0.718779
Epoch 137
Validation auc = 0.716432
Epoch 138
Validation auc = 0.714085
Epoch 139
Loss = 8.7077e-02, PNorm = 55.1698, GNorm = 1.5186, lr_0 = 7.9788e-04
Validation auc = 0.722535
Epoch 140
Validation auc = 0.724883
Epoch 141
Validation auc = 0.723944
Epoch 142
Validation auc = 0.713615
Epoch 143
Validation auc = 0.708451
Epoch 144
Loss = 4.2409e-02, PNorm = 55.1984, GNorm = 1.4375, lr_0 = 7.9106e-04
Validation auc = 0.705634
Epoch 145
Validation auc = 0.709859
Epoch 146
Validation auc = 0.710798
Epoch 147
Validation auc = 0.715962
Epoch 148
Validation auc = 0.716432
Epoch 149
Loss = 1.1499e-02, PNorm = 55.2321, GNorm = 0.3406, lr_0 = 7.8430e-04
Validation auc = 0.718310
Epoch 150
Validation auc = 0.719718
Epoch 151
Validation auc = 0.720188
Epoch 152
Validation auc = 0.722535
Epoch 153
Validation auc = 0.722066
Epoch 154
Loss = 8.3427e-03, PNorm = 55.2673, GNorm = 0.1615, lr_0 = 7.7759e-04
Validation auc = 0.724413
Epoch 155
Validation auc = 0.723474
Epoch 156
Validation auc = 0.723474
Epoch 157
Validation auc = 0.717840
Epoch 158
Validation auc = 0.718310
Epoch 159
Loss = 5.2815e-03, PNorm = 55.2920, GNorm = 0.3601, lr_0 = 7.7095e-04
Validation auc = 0.723944
Epoch 160
Validation auc = 0.727700
Epoch 161
Validation auc = 0.731455
Epoch 162
Validation auc = 0.730516
Epoch 163
Validation auc = 0.730516
Epoch 164
Loss = 9.0262e-03, PNorm = 55.3215, GNorm = 0.7223, lr_0 = 7.6436e-04
Validation auc = 0.725352
Epoch 165
Validation auc = 0.724413
Epoch 166
Validation auc = 0.721596
Epoch 167
Validation auc = 0.719249
Epoch 168
Validation auc = 0.715493
Epoch 169
Loss = 3.6150e-03, PNorm = 55.3487, GNorm = 0.3118, lr_0 = 7.5782e-04
Validation auc = 0.713615
Epoch 170
Validation auc = 0.710798
Epoch 171
Validation auc = 0.710798
Epoch 172
Validation auc = 0.709859
Epoch 173
Validation auc = 0.714554
Epoch 174
Loss = 7.2001e-03, PNorm = 55.3761, GNorm = 0.5445, lr_0 = 7.5134e-04
Validation auc = 0.717371
Epoch 175
Validation auc = 0.717371
Epoch 176
Validation auc = 0.720657
Epoch 177
Validation auc = 0.723005
Epoch 178
Validation auc = 0.721596
Epoch 179
Loss = 1.3275e-02, PNorm = 55.4115, GNorm = 0.5565, lr_0 = 7.4492e-04
Validation auc = 0.717840
Epoch 180
Validation auc = 0.711268
Epoch 181
Validation auc = 0.709390
Epoch 182
Validation auc = 0.710798
Epoch 183
Validation auc = 0.708920
Epoch 184
Loss = 4.8987e-03, PNorm = 55.4503, GNorm = 0.0380, lr_0 = 7.3855e-04
Validation auc = 0.707981
Epoch 185
Validation auc = 0.708920
Epoch 186
Validation auc = 0.711737
Epoch 187
Validation auc = 0.709859
Epoch 188
Validation auc = 0.717371
Epoch 189
Loss = 2.0110e-02, PNorm = 55.4929, GNorm = 0.0573, lr_0 = 7.3224e-04
Validation auc = 0.721596
Epoch 190
Validation auc = 0.722066
Epoch 191
Validation auc = 0.721127
Epoch 192
Validation auc = 0.717840
Epoch 193
Validation auc = 0.715023
Epoch 194
Loss = 2.0576e-02, PNorm = 55.5517, GNorm = 1.6428, lr_0 = 7.2598e-04
Validation auc = 0.709390
Epoch 195
Validation auc = 0.708451
Epoch 196
Validation auc = 0.709390
Epoch 197
Validation auc = 0.710798
Epoch 198
Validation auc = 0.708920
Epoch 199
Loss = 6.1155e-03, PNorm = 55.6103, GNorm = 0.4894, lr_0 = 7.1977e-04
Validation auc = 0.711268
Epoch 200
Validation auc = 0.713146
Epoch 201
Validation auc = 0.712676
Epoch 202
Validation auc = 0.718779
Epoch 203
Validation auc = 0.724883
Epoch 204
Loss = 2.7235e-02, PNorm = 55.6505, GNorm = 1.8640, lr_0 = 7.1362e-04
Validation auc = 0.723005
Epoch 205
Validation auc = 0.714554
Epoch 206
Validation auc = 0.710798
Epoch 207
Validation auc = 0.703756
Epoch 208
Validation auc = 0.703286
Epoch 209
Loss = 7.6940e-03, PNorm = 55.6977, GNorm = 0.2695, lr_0 = 7.0752e-04
Validation auc = 0.703286
Epoch 210
Validation auc = 0.712676
Epoch 211
Validation auc = 0.722066
Epoch 212
Validation auc = 0.721127
Epoch 213
Validation auc = 0.723474
Epoch 214
Loss = 5.8939e-02, PNorm = 55.7573, GNorm = 2.3347, lr_0 = 7.0147e-04
Validation auc = 0.723944
Epoch 215
Validation auc = 0.734742
Epoch 216
Validation auc = 0.738028
Epoch 217
Validation auc = 0.739906
Epoch 218
Validation auc = 0.731455
Epoch 219
Loss = 2.6005e-02, PNorm = 55.8091, GNorm = 1.8464, lr_0 = 6.9548e-04
Validation auc = 0.720657
Epoch 220
Validation auc = 0.713615
Epoch 221
Validation auc = 0.704695
Epoch 222
Validation auc = 0.704695
Epoch 223
Validation auc = 0.712207
Epoch 224
Loss = 1.1332e-03, PNorm = 55.8860, GNorm = 0.0182, lr_0 = 6.8953e-04
Validation auc = 0.712676
Epoch 225
Validation auc = 0.715962
Epoch 226
Validation auc = 0.710798
Epoch 227
Validation auc = 0.705634
Epoch 228
Validation auc = 0.705164
Epoch 229
Loss = 3.3434e-03, PNorm = 55.9390, GNorm = 0.1004, lr_0 = 6.8364e-04
Validation auc = 0.703286
Epoch 230
Validation auc = 0.706103
Epoch 231
Validation auc = 0.706573
Epoch 232
Validation auc = 0.709390
Epoch 233
Validation auc = 0.711268
Epoch 234
Loss = 2.3064e-03, PNorm = 55.9815, GNorm = 0.0492, lr_0 = 6.7779e-04
Validation auc = 0.710798
Epoch 235
Validation auc = 0.710329
Epoch 236
Validation auc = 0.711737
Epoch 237
Validation auc = 0.709859
Epoch 238
Validation auc = 0.713615
Epoch 239
Loss = 5.0423e-02, PNorm = 56.0093, GNorm = 2.5099, lr_0 = 6.7200e-04
Validation auc = 0.710798
Epoch 240
Validation auc = 0.705634
Epoch 241
Validation auc = 0.702817
Epoch 242
Validation auc = 0.699061
Epoch 243
Validation auc = 0.697183
Epoch 244
Loss = 2.1455e-02, PNorm = 56.0311, GNorm = 0.8887, lr_0 = 6.6625e-04
Validation auc = 0.707042
Epoch 245
Validation auc = 0.721127
Epoch 246
Validation auc = 0.726291
Epoch 247
Validation auc = 0.727700
Epoch 248
Validation auc = 0.730516
Epoch 249
Loss = 3.4681e-02, PNorm = 56.0748, GNorm = 1.6497, lr_0 = 6.6056e-04
Validation auc = 0.730986
Epoch 250
Validation auc = 0.727700
Epoch 251
Validation auc = 0.729108
Epoch 252
Validation auc = 0.730047
Epoch 253
Validation auc = 0.730516
Epoch 254
Loss = 1.6911e-03, PNorm = 56.1204, GNorm = 0.1016, lr_0 = 6.5491e-04
Validation auc = 0.732394
Epoch 255
Validation auc = 0.730986
Epoch 256
Validation auc = 0.730986
Epoch 257
Validation auc = 0.730047
Epoch 258
Validation auc = 0.725352
Epoch 259
Loss = 2.2402e-03, PNorm = 56.1567, GNorm = 0.2921, lr_0 = 6.4931e-04
Validation auc = 0.721127
Epoch 260
Validation auc = 0.718779
Epoch 261
Validation auc = 0.724413
Epoch 262
Validation auc = 0.732864
Epoch 263
Validation auc = 0.737559
Epoch 264
Loss = 6.4683e-02, PNorm = 56.1843, GNorm = 2.0436, lr_0 = 6.4376e-04
Validation auc = 0.736150
Epoch 265
Validation auc = 0.731455
Epoch 266
Validation auc = 0.726291
Epoch 267
Validation auc = 0.723005
Epoch 268
Validation auc = 0.716901
Epoch 269
Loss = 2.4325e-03, PNorm = 56.2347, GNorm = 0.0560, lr_0 = 6.3826e-04
Validation auc = 0.712676
Epoch 270
Validation auc = 0.706573
Epoch 271
Validation auc = 0.703286
Epoch 272
Validation auc = 0.703286
Epoch 273
Validation auc = 0.703756
Epoch 274
Loss = 1.2737e-02, PNorm = 56.2765, GNorm = 0.1762, lr_0 = 6.3280e-04
Validation auc = 0.706103
Epoch 275
Validation auc = 0.711268
Epoch 276
Validation auc = 0.714085
Epoch 277
Validation auc = 0.715023
Epoch 278
Validation auc = 0.719718
Epoch 279
Loss = 1.4487e-03, PNorm = 56.3097, GNorm = 0.0471, lr_0 = 6.2739e-04
Validation auc = 0.723474
Epoch 280
Validation auc = 0.726291
Epoch 281
Validation auc = 0.725117
Epoch 282
Validation auc = 0.725117
Epoch 283
Validation auc = 0.725822
Epoch 284
Loss = 4.1908e-03, PNorm = 56.3371, GNorm = 0.5007, lr_0 = 6.2203e-04
Validation auc = 0.725822
Epoch 285
Validation auc = 0.725822
Epoch 286
Validation auc = 0.727700
Epoch 287
Validation auc = 0.726761
Epoch 288
Validation auc = 0.728169
Epoch 289
Loss = 6.7929e-04, PNorm = 56.3587, GNorm = 0.0332, lr_0 = 6.1671e-04
Validation auc = 0.727230
Epoch 290
Validation auc = 0.728169
Epoch 291
Validation auc = 0.727230
Epoch 292
Validation auc = 0.725352
Epoch 293
Validation auc = 0.722066
Epoch 294
Loss = 8.5805e-03, PNorm = 56.3771, GNorm = 1.0276, lr_0 = 6.1144e-04
Validation auc = 0.714554
Epoch 295
Validation auc = 0.710798
Epoch 296
Validation auc = 0.705164
Epoch 297
Validation auc = 0.703286
Epoch 298
Validation auc = 0.704695
Epoch 299
Loss = 8.7438e-04, PNorm = 56.3977, GNorm = 0.0820, lr_0 = 6.0621e-04
Validation auc = 0.707042
Model 0 best validation auc = 0.767606 on epoch 38
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.894040
Ensemble test auc = 0.894040
Fold 3
Splitting data with seed 3
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation auc = 0.783568
Epoch 1
Validation auc = 0.758216
Epoch 2
Validation auc = 0.734742
Epoch 3
Validation auc = 0.757277
Epoch 4
Loss = 4.9149e-01, PNorm = 52.2993, GNorm = 3.4316, lr_0 = 6.5000e-04
Validation auc = 0.726761
Epoch 5
Validation auc = 0.746009
Epoch 6
Validation auc = 0.746009
Epoch 7
Validation auc = 0.758685
Epoch 8
Validation auc = 0.751174
Epoch 9
Loss = 2.1614e-01, PNorm = 52.5547, GNorm = 1.2152, lr_0 = 9.9743e-04
Validation auc = 0.729108
Epoch 10
Validation auc = 0.714554
Epoch 11
Validation auc = 0.704225
Epoch 12
Validation auc = 0.700939
Epoch 13
Validation auc = 0.717840
Epoch 14
Loss = 2.0136e-01, PNorm = 52.8389, GNorm = 2.0497, lr_0 = 9.8890e-04
Validation auc = 0.717840
Epoch 15
Validation auc = 0.725352
Epoch 16
Validation auc = 0.740376
Epoch 17
Validation auc = 0.738967
Epoch 18
Validation auc = 0.731925
Epoch 19
Loss = 1.6321e-01, PNorm = 53.0707, GNorm = 1.6615, lr_0 = 9.8045e-04
Validation auc = 0.760563
Epoch 20
Validation auc = 0.773709
Epoch 21
Validation auc = 0.769953
Epoch 22
Validation auc = 0.776056
Epoch 23
Validation auc = 0.762441
Epoch 24
Loss = 2.3461e-01, PNorm = 53.2607, GNorm = 5.3498, lr_0 = 9.7207e-04
Validation auc = 0.749765
Epoch 25
Validation auc = 0.723944
Epoch 26
Validation auc = 0.729108
Epoch 27
Validation auc = 0.738967
Epoch 28
Validation auc = 0.721596
Epoch 29
Loss = 8.6706e-02, PNorm = 53.4230, GNorm = 0.8821, lr_0 = 9.6376e-04
Validation auc = 0.711737
Epoch 30
Validation auc = 0.715962
Epoch 31
Validation auc = 0.715493
Epoch 32
Validation auc = 0.707512
Epoch 33
Validation auc = 0.696244
Epoch 34
Loss = 6.1636e-02, PNorm = 53.5650, GNorm = 2.1385, lr_0 = 9.5552e-04
Validation auc = 0.698592
Epoch 35
Validation auc = 0.715493
Epoch 36
Validation auc = 0.722535
Epoch 37
Validation auc = 0.723005
Epoch 38
Validation auc = 0.736620
Epoch 39
Loss = 6.0779e-02, PNorm = 53.6776, GNorm = 2.0885, lr_0 = 9.4735e-04
Validation auc = 0.739906
Epoch 40
Validation auc = 0.744131
Epoch 41
Validation auc = 0.737559
Epoch 42
Validation auc = 0.718310
Epoch 43
Validation auc = 0.723005
Epoch 44
Loss = 1.0709e-01, PNorm = 53.7711, GNorm = 1.0068, lr_0 = 9.3925e-04
Validation auc = 0.728169
Epoch 45
Validation auc = 0.745540
Epoch 46
Validation auc = 0.750704
Epoch 47
Validation auc = 0.748826
Epoch 48
Validation auc = 0.746009
Epoch 49
Loss = 1.0455e-01, PNorm = 53.8728, GNorm = 1.6531, lr_0 = 9.3122e-04
Validation auc = 0.736150
Epoch 50
Validation auc = 0.726291
Epoch 51
Validation auc = 0.699061
Epoch 52
Validation auc = 0.702347
Epoch 53
Validation auc = 0.719718
Epoch 54
Loss = 2.0262e-02, PNorm = 53.9737, GNorm = 1.3034, lr_0 = 9.2326e-04
Validation auc = 0.717840
Epoch 55
Validation auc = 0.716432
Epoch 56
Validation auc = 0.719718
Epoch 57
Validation auc = 0.728169
Epoch 58
Validation auc = 0.725352
Epoch 59
Loss = 5.5539e-02, PNorm = 54.0515, GNorm = 2.2314, lr_0 = 9.1537e-04
Validation auc = 0.706103
Epoch 60
Validation auc = 0.696714
Epoch 61
Validation auc = 0.701878
Epoch 62
Validation auc = 0.721596
Epoch 63
Validation auc = 0.731925
Epoch 64
Loss = 3.4311e-02, PNorm = 54.1606, GNorm = 0.3978, lr_0 = 9.0754e-04
Validation auc = 0.738967
Epoch 65
Validation auc = 0.753991
Epoch 66
Validation auc = 0.749296
Epoch 67
Validation auc = 0.746479
Epoch 68
Validation auc = 0.746479
Epoch 69
Loss = 5.7532e-02, PNorm = 54.2883, GNorm = 1.1263, lr_0 = 8.9978e-04
Validation auc = 0.748357
Epoch 70
Validation auc = 0.746948
Epoch 71
Validation auc = 0.745540
Epoch 72
Validation auc = 0.745540
Epoch 73
Validation auc = 0.743662
Epoch 74
Loss = 2.2834e-02, PNorm = 54.4007, GNorm = 0.3745, lr_0 = 8.9209e-04
Validation auc = 0.737559
Epoch 75
Validation auc = 0.731925
Epoch 76
Validation auc = 0.726761
Epoch 77
Validation auc = 0.726761
Epoch 78
Validation auc = 0.729108
Epoch 79
Loss = 1.8261e-02, PNorm = 54.4937, GNorm = 0.8508, lr_0 = 8.8447e-04
Validation auc = 0.732394
Epoch 80
Validation auc = 0.731455
Epoch 81
Validation auc = 0.730516
Epoch 82
Validation auc = 0.733333
Epoch 83
Validation auc = 0.737089
Epoch 84
Loss = 4.0483e-02, PNorm = 54.5572, GNorm = 1.4628, lr_0 = 8.7691e-04
Validation auc = 0.733803
Epoch 85
Validation auc = 0.728169
Epoch 86
Validation auc = 0.738028
Epoch 87
Validation auc = 0.733803
Epoch 88
Validation auc = 0.730986
Epoch 89
Loss = 1.1792e-01, PNorm = 54.6344, GNorm = 2.8214, lr_0 = 8.6941e-04
Validation auc = 0.733803
Epoch 90
Validation auc = 0.719718
Epoch 91
Validation auc = 0.713615
Epoch 92
Validation auc = 0.725352
Epoch 93
Validation auc = 0.743662
Epoch 94
Loss = 2.8005e-02, PNorm = 54.7241, GNorm = 0.9999, lr_0 = 8.6198e-04
Validation auc = 0.754930
Epoch 95
Validation auc = 0.753521
Epoch 96
Validation auc = 0.752113
Epoch 97
Validation auc = 0.748357
Epoch 98
Validation auc = 0.742723
Epoch 99
Loss = 5.9587e-02, PNorm = 54.8107, GNorm = 0.9776, lr_0 = 8.5461e-04
Validation auc = 0.742723
Epoch 100
Validation auc = 0.738498
Epoch 101
Validation auc = 0.733803
Epoch 102
Validation auc = 0.732864
Epoch 103
Validation auc = 0.726761
Epoch 104
Loss = 9.8554e-03, PNorm = 54.8868, GNorm = 0.5324, lr_0 = 8.4730e-04
Validation auc = 0.721127
Epoch 105
Validation auc = 0.727230
Epoch 106
Validation auc = 0.737559
Epoch 107
Validation auc = 0.744131
Epoch 108
Validation auc = 0.748357
Epoch 109
Loss = 1.0072e-02, PNorm = 54.9513, GNorm = 0.4537, lr_0 = 8.4006e-04
Validation auc = 0.750704
Epoch 110
Validation auc = 0.745540
Epoch 111
Validation auc = 0.740376
Epoch 112
Validation auc = 0.726761
Epoch 113
Validation auc = 0.724413
Epoch 114
Loss = 5.4824e-03, PNorm = 55.0121, GNorm = 0.1665, lr_0 = 8.3288e-04
Validation auc = 0.723005
Epoch 115
Validation auc = 0.720657
Epoch 116
Validation auc = 0.720657
Epoch 117
Validation auc = 0.721596
Epoch 118
Validation auc = 0.721362
Epoch 119
Loss = 8.3777e-02, PNorm = 55.0928, GNorm = 1.0234, lr_0 = 8.2576e-04
Validation auc = 0.737559
Epoch 120
Validation auc = 0.746009
Epoch 121
Validation auc = 0.751643
Epoch 122
Validation auc = 0.751643
Epoch 123
Validation auc = 0.750704
Epoch 124
Loss = 3.0125e-02, PNorm = 55.1671, GNorm = 0.2691, lr_0 = 8.1870e-04
Validation auc = 0.752582
Epoch 125
Validation auc = 0.750235
Epoch 126
Validation auc = 0.748826
Epoch 127
Validation auc = 0.748357
Epoch 128
Validation auc = 0.748357
Epoch 129
Loss = 3.3512e-02, PNorm = 55.2259, GNorm = 1.3297, lr_0 = 8.1170e-04
Validation auc = 0.750235
Epoch 130
Validation auc = 0.753052
Epoch 131
Validation auc = 0.750235
Epoch 132
Validation auc = 0.749296
Epoch 133
Validation auc = 0.746479
Epoch 134
Loss = 1.0265e-02, PNorm = 55.2709, GNorm = 0.5671, lr_0 = 8.0476e-04
Validation auc = 0.742723
Epoch 135
Validation auc = 0.736620
Epoch 136
Validation auc = 0.729108
Epoch 137
Validation auc = 0.723474
Epoch 138
Validation auc = 0.710798
Epoch 139
Loss = 9.4771e-03, PNorm = 55.3076, GNorm = 0.3607, lr_0 = 7.9788e-04
Validation auc = 0.704695
Epoch 140
Validation auc = 0.709859
Epoch 141
Validation auc = 0.719249
Epoch 142
Validation auc = 0.722535
Epoch 143
Validation auc = 0.724413
Epoch 144
Loss = 3.4916e-03, PNorm = 55.3449, GNorm = 0.2848, lr_0 = 7.9106e-04
Validation auc = 0.723474
Epoch 145
Validation auc = 0.725352
Epoch 146
Validation auc = 0.722535
Epoch 147
Validation auc = 0.708920
Epoch 148
Validation auc = 0.707042
Epoch 149
Loss = 4.7247e-02, PNorm = 55.3773, GNorm = 1.3930, lr_0 = 7.8430e-04
Validation auc = 0.707981
Epoch 150
Validation auc = 0.712676
Epoch 151
Validation auc = 0.714554
Epoch 152
Validation auc = 0.712676
Epoch 153
Validation auc = 0.717371
Epoch 154
Loss = 4.9824e-03, PNorm = 55.4330, GNorm = 0.3878, lr_0 = 7.7759e-04
Validation auc = 0.718779
Epoch 155
Validation auc = 0.718779
Epoch 156
Validation auc = 0.721127
Epoch 157
Validation auc = 0.725352
Epoch 158
Validation auc = 0.728169
Epoch 159
Loss = 3.1510e-03, PNorm = 55.4813, GNorm = 0.0248, lr_0 = 7.7095e-04
Validation auc = 0.733333
Epoch 160
Validation auc = 0.732864
Epoch 161
Validation auc = 0.733333
Epoch 162
Validation auc = 0.732394
Epoch 163
Validation auc = 0.730516
Epoch 164
Loss = 3.5257e-02, PNorm = 55.5133, GNorm = 0.0439, lr_0 = 7.6436e-04
Validation auc = 0.730986
Epoch 165
Validation auc = 0.730047
Epoch 166
Validation auc = 0.727700
Epoch 167
Validation auc = 0.726291
Epoch 168
Validation auc = 0.724413
Epoch 169
Loss = 8.5574e-03, PNorm = 55.5403, GNorm = 0.0716, lr_0 = 7.5782e-04
Validation auc = 0.724883
Epoch 170
Validation auc = 0.725822
Epoch 171
Validation auc = 0.722066
Epoch 172
Validation auc = 0.720188
Epoch 173
Validation auc = 0.722066
Epoch 174
Loss = 1.9227e-03, PNorm = 55.5640, GNorm = 0.1312, lr_0 = 7.5134e-04
Validation auc = 0.717840
Epoch 175
Validation auc = 0.720657
Epoch 176
Validation auc = 0.727230
Epoch 177
Validation auc = 0.727230
Epoch 178
Validation auc = 0.729577
Epoch 179
Loss = 9.7840e-03, PNorm = 55.5877, GNorm = 0.4281, lr_0 = 7.4492e-04
Validation auc = 0.729108
Epoch 180
Validation auc = 0.729108
Epoch 181
Validation auc = 0.725822
Epoch 182
Validation auc = 0.724413
Epoch 183
Validation auc = 0.727700
Epoch 184
Loss = 2.3478e-02, PNorm = 55.6225, GNorm = 0.2088, lr_0 = 7.3855e-04
Validation auc = 0.718310
Epoch 185
Validation auc = 0.711268
Epoch 186
Validation auc = 0.713146
Epoch 187
Validation auc = 0.717371
Epoch 188
Validation auc = 0.723005
Epoch 189
Loss = 7.0880e-04, PNorm = 55.6970, GNorm = 0.0323, lr_0 = 7.3224e-04
Validation auc = 0.725352
Epoch 190
Validation auc = 0.731455
Epoch 191
Validation auc = 0.726291
Epoch 192
Validation auc = 0.722535
Epoch 193
Validation auc = 0.707512
Epoch 194
Loss = 3.2887e-03, PNorm = 55.7581, GNorm = 0.1796, lr_0 = 7.2598e-04
Validation auc = 0.700704
Epoch 195
Validation auc = 0.700704
Epoch 196
Validation auc = 0.713146
Epoch 197
Validation auc = 0.721831
Epoch 198
Validation auc = 0.727230
Epoch 199
Loss = 4.2172e-03, PNorm = 55.8264, GNorm = 0.1586, lr_0 = 7.1977e-04
Validation auc = 0.725352
Epoch 200
Validation auc = 0.724413
Epoch 201
Validation auc = 0.725352
Epoch 202
Validation auc = 0.721596
Epoch 203
Validation auc = 0.722066
Epoch 204
Loss = 9.6756e-03, PNorm = 55.8768, GNorm = 0.5007, lr_0 = 7.1362e-04
Validation auc = 0.710329
Epoch 205
Validation auc = 0.701878
Epoch 206
Validation auc = 0.701878
Epoch 207
Validation auc = 0.702113
Epoch 208
Validation auc = 0.701643
Epoch 209
Loss = 3.2855e-03, PNorm = 55.9269, GNorm = 0.0033, lr_0 = 7.0752e-04
Validation auc = 0.696714
Epoch 210
Validation auc = 0.699061
Epoch 211
Validation auc = 0.702347
Epoch 212
Validation auc = 0.704225
Epoch 213
Validation auc = 0.706573
Epoch 214
Loss = 1.3534e-03, PNorm = 55.9712, GNorm = 0.1145, lr_0 = 7.0147e-04
Validation auc = 0.708451
Epoch 215
Validation auc = 0.707981
Epoch 216
Validation auc = 0.709390
Epoch 217
Validation auc = 0.711737
Epoch 218
Validation auc = 0.715493
Epoch 219
Loss = 5.4265e-03, PNorm = 56.0025, GNorm = 0.1872, lr_0 = 6.9548e-04
Validation auc = 0.715962
Epoch 220
Validation auc = 0.717840
Epoch 221
Validation auc = 0.712676
Epoch 222
Validation auc = 0.708920
Epoch 223
Validation auc = 0.706338
Epoch 224
Loss = 7.8370e-02, PNorm = 56.0354, GNorm = 2.0761, lr_0 = 6.8953e-04
Validation auc = 0.711972
Epoch 225
Validation auc = 0.719718
Epoch 226
Validation auc = 0.724883
Epoch 227
Validation auc = 0.725822
Epoch 228
Validation auc = 0.723944
Epoch 229
Loss = 4.5397e-02, PNorm = 56.0740, GNorm = 2.9988, lr_0 = 6.8364e-04
Validation auc = 0.721831
Epoch 230
Validation auc = 0.716197
Epoch 231
Validation auc = 0.716197
Epoch 232
Validation auc = 0.719014
Epoch 233
Validation auc = 0.725352
Epoch 234
Loss = 7.1777e-03, PNorm = 56.1198, GNorm = 0.3586, lr_0 = 6.7779e-04
Validation auc = 0.732394
Epoch 235
Validation auc = 0.732394
Epoch 236
Validation auc = 0.735681
Epoch 237
Validation auc = 0.735681
Epoch 238
Validation auc = 0.734272
Epoch 239
Loss = 2.3288e-02, PNorm = 56.1641, GNorm = 1.1499, lr_0 = 6.7200e-04
Validation auc = 0.733803
Epoch 240
Validation auc = 0.730047
Epoch 241
Validation auc = 0.718310
Epoch 242
Validation auc = 0.716432
Epoch 243
Validation auc = 0.718310
Epoch 244
Loss = 1.0733e-03, PNorm = 56.2022, GNorm = 0.0366, lr_0 = 6.6625e-04
Validation auc = 0.719718
Epoch 245
Validation auc = 0.718310
Epoch 246
Validation auc = 0.718545
Epoch 247
Validation auc = 0.718545
Epoch 248
Validation auc = 0.718779
Epoch 249
Loss = 7.5053e-03, PNorm = 56.2405, GNorm = 0.0481, lr_0 = 6.6056e-04
Validation auc = 0.724883
Epoch 250
Validation auc = 0.723005
Epoch 251
Validation auc = 0.724413
Epoch 252
Validation auc = 0.723005
Epoch 253
Validation auc = 0.724413
Epoch 254
Loss = 3.4932e-03, PNorm = 56.2846, GNorm = 0.3413, lr_0 = 6.5491e-04
Validation auc = 0.722535
Epoch 255
Validation auc = 0.717606
Epoch 256
Validation auc = 0.718545
Epoch 257
Validation auc = 0.721831
Epoch 258
Validation auc = 0.726291
Epoch 259
Loss = 3.8482e-02, PNorm = 56.3179, GNorm = 1.6184, lr_0 = 6.4931e-04
Validation auc = 0.722535
Epoch 260
Validation auc = 0.714085
Epoch 261
Validation auc = 0.707512
Epoch 262
Validation auc = 0.709390
Epoch 263
Validation auc = 0.708451
Epoch 264
Loss = 1.5543e-02, PNorm = 56.3603, GNorm = 0.0955, lr_0 = 6.4376e-04
Validation auc = 0.709390
Epoch 265
Validation auc = 0.705634
Epoch 266
Validation auc = 0.706103
Epoch 267
Validation auc = 0.705164
Epoch 268
Validation auc = 0.703052
Epoch 269
Loss = 7.0215e-03, PNorm = 56.4029, GNorm = 0.0528, lr_0 = 6.3826e-04
Validation auc = 0.703991
Epoch 270
Validation auc = 0.704225
Epoch 271
Validation auc = 0.706573
Epoch 272
Validation auc = 0.706808
Epoch 273
Validation auc = 0.713615
Epoch 274
Loss = 3.3187e-03, PNorm = 56.4409, GNorm = 0.0820, lr_0 = 6.3280e-04
Validation auc = 0.721127
Epoch 275
Validation auc = 0.727700
Epoch 276
Validation auc = 0.730047
Epoch 277
Validation auc = 0.730986
Epoch 278
Validation auc = 0.729577
Epoch 279
Loss = 5.9977e-04, PNorm = 56.4742, GNorm = 0.0387, lr_0 = 6.2739e-04
Validation auc = 0.728638
Epoch 280
Validation auc = 0.728169
Epoch 281
Validation auc = 0.729108
Epoch 282
Validation auc = 0.719718
Epoch 283
Validation auc = 0.713615
Epoch 284
Loss = 1.2533e-02, PNorm = 56.5048, GNorm = 0.1231, lr_0 = 6.2203e-04
Validation auc = 0.713146
Epoch 285
Validation auc = 0.715493
Epoch 286
Validation auc = 0.717840
Epoch 287
Validation auc = 0.718310
Epoch 288
Validation auc = 0.722535
Epoch 289
Loss = 4.9716e-04, PNorm = 56.5509, GNorm = 0.0193, lr_0 = 6.1671e-04
Validation auc = 0.725822
Epoch 290
Validation auc = 0.727230
Epoch 291
Validation auc = 0.726761
Epoch 292
Validation auc = 0.727700
Epoch 293
Validation auc = 0.729108
Epoch 294
Loss = 3.1960e-03, PNorm = 56.5836, GNorm = 0.1963, lr_0 = 6.1144e-04
Validation auc = 0.729577
Epoch 295
Validation auc = 0.729577
Epoch 296
Validation auc = 0.730516
Epoch 297
Validation auc = 0.732394
Epoch 298
Validation auc = 0.732864
Epoch 299
Loss = 1.0918e-03, PNorm = 56.6122, GNorm = 0.0709, lr_0 = 6.0621e-04
Validation auc = 0.731455
Model 0 best validation auc = 0.783568 on epoch 0
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.791692
Ensemble test auc = 0.791692
Fold 4
Splitting data with seed 4
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation auc = 0.682629
Epoch 1
Validation auc = 0.714085
Epoch 2
Validation auc = 0.712207
Epoch 3
Validation auc = 0.746009
Epoch 4
Loss = 6.3780e-01, PNorm = 52.2963, GNorm = 6.4829, lr_0 = 6.5000e-04
Validation auc = 0.736620
Epoch 5
Validation auc = 0.741315
Epoch 6
Validation auc = 0.762441
Epoch 7
Validation auc = 0.757746
Epoch 8
Validation auc = 0.740845
Epoch 9
Loss = 2.4389e-01, PNorm = 52.5413, GNorm = 1.0832, lr_0 = 9.9743e-04
Validation auc = 0.749765
Epoch 10
Validation auc = 0.738967
Epoch 11
Validation auc = 0.707042
Epoch 12
Validation auc = 0.718310
Epoch 13
Validation auc = 0.706103
Epoch 14
Loss = 2.0630e-01, PNorm = 52.8260, GNorm = 1.8653, lr_0 = 9.8890e-04
Validation auc = 0.696244
Epoch 15
Validation auc = 0.705164
Epoch 16
Validation auc = 0.724413
Epoch 17
Validation auc = 0.732394
Epoch 18
Validation auc = 0.733333
Epoch 19
Loss = 1.3292e-01, PNorm = 53.0526, GNorm = 1.0030, lr_0 = 9.8045e-04
Validation auc = 0.732394
Epoch 20
Validation auc = 0.738967
Epoch 21
Validation auc = 0.737089
Epoch 22
Validation auc = 0.735211
Epoch 23
Validation auc = 0.730516
Epoch 24
Loss = 1.4989e-01, PNorm = 53.2189, GNorm = 1.3892, lr_0 = 9.7207e-04
Validation auc = 0.722066
Epoch 25
Validation auc = 0.707042
Epoch 26
Validation auc = 0.707512
Epoch 27
Validation auc = 0.726761
Epoch 28
Validation auc = 0.728638
Epoch 29
Loss = 8.9433e-02, PNorm = 53.3514, GNorm = 0.7518, lr_0 = 9.6376e-04
Validation auc = 0.721596
Epoch 30
Validation auc = 0.710329
Epoch 31
Validation auc = 0.702817
Epoch 32
Validation auc = 0.707042
Epoch 33
Validation auc = 0.704695
Epoch 34
Loss = 9.5897e-02, PNorm = 53.4822, GNorm = 1.0555, lr_0 = 9.5552e-04
Validation auc = 0.706103
Epoch 35
Validation auc = 0.712207
Epoch 36
Validation auc = 0.720657
Epoch 37
Validation auc = 0.684038
Epoch 38
Validation auc = 0.659155
Epoch 39
Loss = 6.5167e-02, PNorm = 53.6111, GNorm = 0.4728, lr_0 = 9.4735e-04
Validation auc = 0.696714
Epoch 40
Validation auc = 0.700939
Epoch 41
Validation auc = 0.692019
Epoch 42
Validation auc = 0.694366
Epoch 43
Validation auc = 0.722066
Epoch 44
Loss = 2.8594e-02, PNorm = 53.7460, GNorm = 0.7018, lr_0 = 9.3925e-04
Validation auc = 0.749296
Epoch 45
Validation auc = 0.752582
Epoch 46
Validation auc = 0.742254
Epoch 47
Validation auc = 0.733333
Epoch 48
Validation auc = 0.725352
Epoch 49
Loss = 6.1453e-02, PNorm = 53.8741, GNorm = 1.4829, lr_0 = 9.3122e-04
Validation auc = 0.721127
Epoch 50
Validation auc = 0.719249
Epoch 51
Validation auc = 0.720657
Epoch 52
Validation auc = 0.721596
Epoch 53
Validation auc = 0.709859
Epoch 54
Loss = 3.6352e-02, PNorm = 53.9858, GNorm = 0.4294, lr_0 = 9.2326e-04
Validation auc = 0.685915
Epoch 55
Validation auc = 0.695775
Epoch 56
Validation auc = 0.738498
Epoch 57
Validation auc = 0.744131
Epoch 58
Validation auc = 0.743192
Epoch 59
Loss = 2.2716e-01, PNorm = 54.1227, GNorm = 3.3084, lr_0 = 9.1537e-04
Validation auc = 0.748357
Epoch 60
Validation auc = 0.755869
Epoch 61
Validation auc = 0.758685
Epoch 62
Validation auc = 0.747887
Epoch 63
Validation auc = 0.732864
Epoch 64
Loss = 7.9292e-02, PNorm = 54.2624, GNorm = 1.5528, lr_0 = 9.0754e-04
Validation auc = 0.726291
Epoch 65
Validation auc = 0.724883
Epoch 66
Validation auc = 0.730516
Epoch 67
Validation auc = 0.733333
Epoch 68
Validation auc = 0.735211
Epoch 69
Loss = 2.5532e-02, PNorm = 54.3854, GNorm = 0.3832, lr_0 = 8.9978e-04
Validation auc = 0.730516
Epoch 70
Validation auc = 0.726761
Epoch 71
Validation auc = 0.721127
Epoch 72
Validation auc = 0.725352
Epoch 73
Validation auc = 0.725352
Epoch 74
Loss = 3.4129e-02, PNorm = 54.4701, GNorm = 0.7205, lr_0 = 8.9209e-04
Validation auc = 0.725822
Epoch 75
Validation auc = 0.726761
Epoch 76
Validation auc = 0.728169
Epoch 77
Validation auc = 0.717840
Epoch 78
Validation auc = 0.708451
Epoch 79
Loss = 7.1189e-03, PNorm = 54.5399, GNorm = 0.1938, lr_0 = 8.8447e-04
Validation auc = 0.700469
Epoch 80
Validation auc = 0.706573
Epoch 81
Validation auc = 0.712676
Epoch 82
Validation auc = 0.719718
Epoch 83
Validation auc = 0.723005
Epoch 84
Loss = 1.1705e-02, PNorm = 54.5999, GNorm = 0.6070, lr_0 = 8.7691e-04
Validation auc = 0.721127
Epoch 85
Validation auc = 0.720188
Epoch 86
Validation auc = 0.718779
Epoch 87
Validation auc = 0.713615
Epoch 88
Validation auc = 0.710798
Epoch 89
Loss = 2.8040e-02, PNorm = 54.6572, GNorm = 1.3397, lr_0 = 8.6941e-04
Validation auc = 0.708920
Epoch 90
Validation auc = 0.703756
Epoch 91
Validation auc = 0.710329
Epoch 92
Validation auc = 0.714085
Epoch 93
Validation auc = 0.715962
Epoch 94
Loss = 3.8225e-02, PNorm = 54.7141, GNorm = 1.4546, lr_0 = 8.6198e-04
Validation auc = 0.713146
Epoch 95
Validation auc = 0.710798
Epoch 96
Validation auc = 0.709390
Epoch 97
Validation auc = 0.725822
Epoch 98
Validation auc = 0.735681
Epoch 99
Loss = 4.6730e-02, PNorm = 54.7744, GNorm = 1.6542, lr_0 = 8.5461e-04
Validation auc = 0.734742
Epoch 100
Validation auc = 0.731925
Epoch 101
Validation auc = 0.730516
Epoch 102
Validation auc = 0.730516
Epoch 103
Validation auc = 0.729108
Epoch 104
Loss = 2.1426e-02, PNorm = 54.8296, GNorm = 1.0351, lr_0 = 8.4730e-04
Validation auc = 0.726761
Epoch 105
Validation auc = 0.721596
Epoch 106
Validation auc = 0.713146
Epoch 107
Validation auc = 0.709859
Epoch 108
Validation auc = 0.706103
Epoch 109
Loss = 4.8420e-02, PNorm = 54.8888, GNorm = 1.2174, lr_0 = 8.4006e-04
Validation auc = 0.711268
Epoch 110
Validation auc = 0.711737
Epoch 111
Validation auc = 0.711737
Epoch 112
Validation auc = 0.716432
Epoch 113
Validation auc = 0.723944
Epoch 114
Loss = 5.9994e-03, PNorm = 54.9487, GNorm = 0.1198, lr_0 = 8.3288e-04
Validation auc = 0.727230
Epoch 115
Validation auc = 0.728638
Epoch 116
Validation auc = 0.728638
Epoch 117
Validation auc = 0.723944
Epoch 118
Validation auc = 0.724883
Epoch 119
Loss = 2.5858e-02, PNorm = 55.0035, GNorm = 0.4506, lr_0 = 8.2576e-04
Validation auc = 0.724883
Epoch 120
Validation auc = 0.720657
Epoch 121
Validation auc = 0.717840
Epoch 122
Validation auc = 0.719249
Epoch 123
Validation auc = 0.716432
Epoch 124
Loss = 2.4252e-02, PNorm = 55.0617, GNorm = 0.0541, lr_0 = 8.1870e-04
Validation auc = 0.716901
Epoch 125
Validation auc = 0.716901
Epoch 126
Validation auc = 0.718779
Epoch 127
Validation auc = 0.723944
Epoch 128
Validation auc = 0.729108
Epoch 129
Loss = 6.9525e-02, PNorm = 55.1082, GNorm = 1.0493, lr_0 = 8.1170e-04
Validation auc = 0.722066
Epoch 130
Validation auc = 0.713615
Epoch 131
Validation auc = 0.709859
Epoch 132
Validation auc = 0.706573
Epoch 133
Validation auc = 0.703286
Epoch 134
Loss = 9.1824e-03, PNorm = 55.1620, GNorm = 0.0956, lr_0 = 8.0476e-04
Validation auc = 0.702347
Epoch 135
Validation auc = 0.701408
Epoch 136
Validation auc = 0.703756
Epoch 137
Validation auc = 0.705634
Epoch 138
Validation auc = 0.704695
Epoch 139
Loss = 6.2438e-03, PNorm = 55.1996, GNorm = 0.1346, lr_0 = 7.9788e-04
Validation auc = 0.697653
Epoch 140
Validation auc = 0.692019
Epoch 141
Validation auc = 0.699531
Epoch 142
Validation auc = 0.703286
Epoch 143
Validation auc = 0.706573
Epoch 144
Loss = 2.3597e-02, PNorm = 55.2552, GNorm = 0.2974, lr_0 = 7.9106e-04
Validation auc = 0.714085
Epoch 145
Validation auc = 0.715493
Epoch 146
Validation auc = 0.716901
Epoch 147
Validation auc = 0.719249
Epoch 148
Validation auc = 0.715962
Epoch 149
Loss = 8.5369e-03, PNorm = 55.3131, GNorm = 0.5039, lr_0 = 7.8430e-04
Validation auc = 0.711268
Epoch 150
Validation auc = 0.703756
Epoch 151
Validation auc = 0.697183
Epoch 152
Validation auc = 0.690610
Epoch 153
Validation auc = 0.683099
Epoch 154
Loss = 2.4232e-03, PNorm = 55.3660, GNorm = 0.1366, lr_0 = 7.7759e-04
Validation auc = 0.679343
Epoch 155
Validation auc = 0.677934
Epoch 156
Validation auc = 0.683099
Epoch 157
Validation auc = 0.690141
Epoch 158
Validation auc = 0.707042
Epoch 159
Loss = 2.4647e-02, PNorm = 55.4059, GNorm = 0.3770, lr_0 = 7.7095e-04
Validation auc = 0.723944
Epoch 160
Validation auc = 0.726291
Epoch 161
Validation auc = 0.730516
Epoch 162
Validation auc = 0.728638
Epoch 163
Validation auc = 0.728638
Epoch 164
Loss = 1.7008e-02, PNorm = 55.4519, GNorm = 0.0615, lr_0 = 7.6436e-04
Validation auc = 0.729108
Epoch 165
Validation auc = 0.727700
Epoch 166
Validation auc = 0.726761
Epoch 167
Validation auc = 0.725822
Epoch 168
Validation auc = 0.723005
Epoch 169
Loss = 6.9691e-03, PNorm = 55.5182, GNorm = 0.4579, lr_0 = 7.5782e-04
Validation auc = 0.719249
Epoch 170
Validation auc = 0.717840
Epoch 171
Validation auc = 0.716901
Epoch 172
Validation auc = 0.708451
Epoch 173
Validation auc = 0.702582
Epoch 174
Loss = 7.0247e-02, PNorm = 55.5739, GNorm = 1.0004, lr_0 = 7.5134e-04
Validation auc = 0.704225
Epoch 175
Validation auc = 0.714085
Epoch 176
Validation auc = 0.723944
Epoch 177
Validation auc = 0.729108
Epoch 178
Validation auc = 0.719249
Epoch 179
Loss = 7.7160e-03, PNorm = 55.6471, GNorm = 0.4972, lr_0 = 7.4492e-04
Validation auc = 0.721127
Epoch 180
Validation auc = 0.718310
Epoch 181
Validation auc = 0.722066
Epoch 182
Validation auc = 0.724648
Epoch 183
Validation auc = 0.730516
Epoch 184
Loss = 3.2782e-02, PNorm = 55.7075, GNorm = 1.1755, lr_0 = 7.3855e-04
Validation auc = 0.736620
Epoch 185
Validation auc = 0.736150
Epoch 186
Validation auc = 0.735211
Epoch 187
Validation auc = 0.736150
Epoch 188
Validation auc = 0.733333
Epoch 189
Loss = 1.0883e-02, PNorm = 55.7552, GNorm = 0.0511, lr_0 = 7.3224e-04
Validation auc = 0.732394
Epoch 190
Validation auc = 0.729577
Epoch 191
Validation auc = 0.729577
Epoch 192
Validation auc = 0.732864
Epoch 193
Validation auc = 0.736150
Epoch 194
Loss = 4.5122e-02, PNorm = 55.7852, GNorm = 1.3914, lr_0 = 7.2598e-04
Validation auc = 0.730986
Epoch 195
Validation auc = 0.730047
Epoch 196
Validation auc = 0.724883
Epoch 197
Validation auc = 0.723005
Epoch 198
Validation auc = 0.718310
Epoch 199
Loss = 1.9434e-02, PNorm = 55.8222, GNorm = 0.1745, lr_0 = 7.1977e-04
Validation auc = 0.717840
Epoch 200
Validation auc = 0.715493
Epoch 201
Validation auc = 0.716432
Epoch 202
Validation auc = 0.718779
Epoch 203
Validation auc = 0.718779
Epoch 204
Loss = 1.0057e-02, PNorm = 55.8623, GNorm = 1.3793, lr_0 = 7.1362e-04
Validation auc = 0.720657
Epoch 205
Validation auc = 0.726761
Epoch 206
Validation auc = 0.732394
Epoch 207
Validation auc = 0.732864
Epoch 208
Validation auc = 0.735211
Epoch 209
Loss = 1.0118e-03, PNorm = 55.9052, GNorm = 0.0519, lr_0 = 7.0752e-04
Validation auc = 0.730986
Epoch 210
Validation auc = 0.720188
Epoch 211
Validation auc = 0.716432
Epoch 212
Validation auc = 0.719718
Epoch 213
Validation auc = 0.721596
Epoch 214
Loss = 3.9113e-03, PNorm = 55.9566, GNorm = 0.0489, lr_0 = 7.0147e-04
Validation auc = 0.722066
Epoch 215
Validation auc = 0.724883
Epoch 216
Validation auc = 0.729108
Epoch 217
Validation auc = 0.731455
Epoch 218
Validation auc = 0.731925
Epoch 219
Loss = 1.8058e-03, PNorm = 55.9993, GNorm = 0.0592, lr_0 = 6.9548e-04
Validation auc = 0.732864
Epoch 220
Validation auc = 0.733333
Epoch 221
Validation auc = 0.734272
Epoch 222
Validation auc = 0.734742
Epoch 223
Validation auc = 0.733803
Epoch 224
Loss = 2.7369e-03, PNorm = 56.0217, GNorm = 0.0941, lr_0 = 6.8953e-04
Validation auc = 0.733803
Epoch 225
Validation auc = 0.732864
Epoch 226
Validation auc = 0.732864
Epoch 227
Validation auc = 0.733333
Epoch 228
Validation auc = 0.734742
Epoch 229
Loss = 4.4611e-03, PNorm = 56.0365, GNorm = 0.4649, lr_0 = 6.8364e-04
Validation auc = 0.732394
Epoch 230
Validation auc = 0.730516
Epoch 231
Validation auc = 0.729108
Epoch 232
Validation auc = 0.727230
Epoch 233
Validation auc = 0.726291
Epoch 234
Loss = 1.6412e-02, PNorm = 56.0577, GNorm = 1.5786, lr_0 = 6.7779e-04
Validation auc = 0.716901
Epoch 235
Validation auc = 0.710798
Epoch 236
Validation auc = 0.706103
Epoch 237
Validation auc = 0.698122
Epoch 238
Validation auc = 0.716432
Epoch 239
Loss = 2.7112e-03, PNorm = 56.0880, GNorm = 0.1360, lr_0 = 6.7200e-04
Validation auc = 0.730516
Epoch 240
Validation auc = 0.733333
Epoch 241
Validation auc = 0.742254
Epoch 242
Validation auc = 0.750235
Epoch 243
Validation auc = 0.747418
Epoch 244
Loss = 9.2302e-03, PNorm = 56.1551, GNorm = 0.9214, lr_0 = 6.6625e-04
Validation auc = 0.738967
Epoch 245
Validation auc = 0.734272
Epoch 246
Validation auc = 0.732160
Epoch 247
Validation auc = 0.731690
Epoch 248
Validation auc = 0.730282
Epoch 249
Loss = 2.8010e-03, PNorm = 56.2101, GNorm = 0.2781, lr_0 = 6.6056e-04
Validation auc = 0.733099
Epoch 250
Validation auc = 0.735211
Epoch 251
Validation auc = 0.739437
Epoch 252
Validation auc = 0.742254
Epoch 253
Validation auc = 0.743192
Epoch 254
Loss = 2.2008e-03, PNorm = 56.2474, GNorm = 0.1503, lr_0 = 6.5491e-04
Validation auc = 0.744131
Epoch 255
Validation auc = 0.743662
Epoch 256
Validation auc = 0.742723
Epoch 257
Validation auc = 0.741784
Epoch 258
Validation auc = 0.739906
Epoch 259
Loss = 5.0357e-04, PNorm = 56.2722, GNorm = 0.0332, lr_0 = 6.4931e-04
Validation auc = 0.733803
Epoch 260
Validation auc = 0.730047
Epoch 261
Validation auc = 0.726761
Epoch 262
Validation auc = 0.723474
Epoch 263
Validation auc = 0.723474
Epoch 264
Loss = 4.5530e-02, PNorm = 56.2883, GNorm = 2.4182, lr_0 = 6.4376e-04
Validation auc = 0.728169
Epoch 265
Validation auc = 0.731925
Epoch 266
Validation auc = 0.734272
Epoch 267
Validation auc = 0.736150
Epoch 268
Validation auc = 0.736620
Epoch 269
Loss = 7.8864e-04, PNorm = 56.3246, GNorm = 0.0384, lr_0 = 6.3826e-04
Validation auc = 0.735681
Epoch 270
Validation auc = 0.733803
Epoch 271
Validation auc = 0.731925
Epoch 272
Validation auc = 0.732394
Epoch 273
Validation auc = 0.729577
Epoch 274
Loss = 1.7936e-03, PNorm = 56.3534, GNorm = 0.0397, lr_0 = 6.3280e-04
Validation auc = 0.729108
Epoch 275
Validation auc = 0.728638
Epoch 276
Validation auc = 0.728638
Epoch 277
Validation auc = 0.730516
Epoch 278
Validation auc = 0.732864
Epoch 279
Loss = 2.4375e-03, PNorm = 56.3827, GNorm = 0.1564, lr_0 = 6.2739e-04
Validation auc = 0.733333
Epoch 280
Validation auc = 0.732864
Epoch 281
Validation auc = 0.730986
Epoch 282
Validation auc = 0.731925
Epoch 283
Validation auc = 0.730047
Epoch 284
Loss = 3.4671e-04, PNorm = 56.4086, GNorm = 0.0263, lr_0 = 6.2203e-04
Validation auc = 0.727700
Epoch 285
Validation auc = 0.730516
Epoch 286
Validation auc = 0.730047
Epoch 287
Validation auc = 0.726761
Epoch 288
Validation auc = 0.721127
Epoch 289
Loss = 1.1176e-02, PNorm = 56.4407, GNorm = 0.2095, lr_0 = 6.1671e-04
Validation auc = 0.717840
Epoch 290
Validation auc = 0.717840
Epoch 291
Validation auc = 0.716432
Epoch 292
Validation auc = 0.718310
Epoch 293
Validation auc = 0.719249
Epoch 294
Loss = 2.1600e-04, PNorm = 56.4625, GNorm = 0.0108, lr_0 = 6.1144e-04
Validation auc = 0.721596
Epoch 295
Validation auc = 0.722066
Epoch 296
Validation auc = 0.726291
Epoch 297
Validation auc = 0.731925
Epoch 298
Validation auc = 0.735211
Epoch 299
Loss = 9.8941e-04, PNorm = 56.4710, GNorm = 0.0924, lr_0 = 6.0621e-04
Validation auc = 0.735211
Model 0 best validation auc = 0.762441 on epoch 6
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.931367
Ensemble test auc = 0.931367
Fold 5
Splitting data with seed 5
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation auc = 0.689202
Epoch 1
Validation auc = 0.672770
Epoch 2
Validation auc = 0.700939
Epoch 3
Validation auc = 0.734272
Epoch 4
Loss = 4.7510e-01, PNorm = 52.3054, GNorm = 3.8084, lr_0 = 6.5000e-04
Validation auc = 0.749296
Epoch 5
Validation auc = 0.748826
Epoch 6
Validation auc = 0.703286
Epoch 7
Validation auc = 0.736150
Epoch 8
Validation auc = 0.728638
Epoch 9
Loss = 4.4363e-01, PNorm = 52.5752, GNorm = 6.6093, lr_0 = 9.9743e-04
Validation auc = 0.750704
Epoch 10
Validation auc = 0.762911
Epoch 11
Validation auc = 0.722066
Epoch 12
Validation auc = 0.697183
Epoch 13
Validation auc = 0.718310
Epoch 14
Loss = 2.5415e-01, PNorm = 52.8781, GNorm = 2.1103, lr_0 = 9.8890e-04
Validation auc = 0.721127
Epoch 15
Validation auc = 0.708920
Epoch 16
Validation auc = 0.722066
Epoch 17
Validation auc = 0.734742
Epoch 18
Validation auc = 0.734272
Epoch 19
Loss = 1.5476e-01, PNorm = 53.1264, GNorm = 1.2449, lr_0 = 9.8045e-04
Validation auc = 0.732394
Epoch 20
Validation auc = 0.720657
Epoch 21
Validation auc = 0.729577
Epoch 22
Validation auc = 0.740376
Epoch 23
Validation auc = 0.733803
Epoch 24
Loss = 1.7071e-01, PNorm = 53.3097, GNorm = 1.0675, lr_0 = 9.7207e-04
Validation auc = 0.724883
Epoch 25
Validation auc = 0.720657
Epoch 26
Validation auc = 0.722535
Epoch 27
Validation auc = 0.718779
Epoch 28
Validation auc = 0.710798
Epoch 29
Loss = 9.3269e-02, PNorm = 53.4492, GNorm = 1.1293, lr_0 = 9.6376e-04
Validation auc = 0.699061
Epoch 30
Validation auc = 0.704225
Epoch 31
Validation auc = 0.711737
Epoch 32
Validation auc = 0.718779
Epoch 33
Validation auc = 0.728638
Epoch 34
Loss = 2.8727e-02, PNorm = 53.5684, GNorm = 0.6137, lr_0 = 9.5552e-04
Validation auc = 0.734272
Epoch 35
Validation auc = 0.736620
Epoch 36
Validation auc = 0.730986
Epoch 37
Validation auc = 0.727700
Epoch 38
Validation auc = 0.726761
Epoch 39
Loss = 6.7664e-02, PNorm = 53.6633, GNorm = 0.3070, lr_0 = 9.4735e-04
Validation auc = 0.723944
Epoch 40
Validation auc = 0.722535
Epoch 41
Validation auc = 0.713146
Epoch 42
Validation auc = 0.712207
Epoch 43
Validation auc = 0.715962
Epoch 44
Loss = 8.3234e-02, PNorm = 53.7502, GNorm = 1.1830, lr_0 = 9.3925e-04
Validation auc = 0.722066
Epoch 45
Validation auc = 0.726291
Epoch 46
Validation auc = 0.732394
Epoch 47
Validation auc = 0.734742
Epoch 48
Validation auc = 0.730047
Epoch 49
Loss = 3.3171e-02, PNorm = 53.8333, GNorm = 0.6459, lr_0 = 9.3122e-04
Validation auc = 0.727230
Epoch 50
Validation auc = 0.722066
Epoch 51
Validation auc = 0.714085
Epoch 52
Validation auc = 0.719249
Epoch 53
Validation auc = 0.724413
Epoch 54
Loss = 2.5386e-02, PNorm = 53.9208, GNorm = 1.7044, lr_0 = 9.2326e-04
Validation auc = 0.725822
Epoch 55
Validation auc = 0.724413
Epoch 56
Validation auc = 0.706103
Epoch 57
Validation auc = 0.676995
Epoch 58
Validation auc = 0.673709
Epoch 59
Loss = 1.1008e-01, PNorm = 54.0142, GNorm = 0.1745, lr_0 = 9.1537e-04
Validation auc = 0.723474
Epoch 60
Validation auc = 0.727230
Epoch 61
Validation auc = 0.717840
Epoch 62
Validation auc = 0.714085
Epoch 63
Validation auc = 0.713615
Epoch 64
Loss = 7.1879e-02, PNorm = 54.1179, GNorm = 0.1006, lr_0 = 9.0754e-04
Validation auc = 0.723944
Epoch 65
Validation auc = 0.740376
Epoch 66
Validation auc = 0.752113
Epoch 67
Validation auc = 0.759624
Epoch 68
Validation auc = 0.758685
Epoch 69
Loss = 1.7937e-02, PNorm = 54.2158, GNorm = 0.3696, lr_0 = 8.9978e-04
Validation auc = 0.757746
Epoch 70
Validation auc = 0.751643
Epoch 71
Validation auc = 0.742723
Epoch 72
Validation auc = 0.741784
Epoch 73
Validation auc = 0.730516
Epoch 74
Loss = 4.2216e-02, PNorm = 54.3158, GNorm = 1.7942, lr_0 = 8.9209e-04
Validation auc = 0.727230
Epoch 75
Validation auc = 0.729577
Epoch 76
Validation auc = 0.732394
Epoch 77
Validation auc = 0.735211
Epoch 78
Validation auc = 0.731455
Epoch 79
Loss = 1.0366e-02, PNorm = 54.4296, GNorm = 0.3058, lr_0 = 8.8447e-04
Validation auc = 0.725352
Epoch 80
Validation auc = 0.720188
Epoch 81
Validation auc = 0.721127
Epoch 82
Validation auc = 0.722066
Epoch 83
Validation auc = 0.723474
Epoch 84
Loss = 7.6613e-02, PNorm = 54.5131, GNorm = 1.4013, lr_0 = 8.7691e-04
Validation auc = 0.713615
Epoch 85
Validation auc = 0.702817
Epoch 86
Validation auc = 0.691080
Epoch 87
Validation auc = 0.696244
Epoch 88
Validation auc = 0.703756
Epoch 89
Loss = 2.0721e-02, PNorm = 54.6045, GNorm = 0.5943, lr_0 = 8.6941e-04
Validation auc = 0.707981
Epoch 90
Validation auc = 0.712207
Epoch 91
Validation auc = 0.719249
Epoch 92
Validation auc = 0.720188
Epoch 93
Validation auc = 0.712207
Epoch 94
Loss = 1.2228e-01, PNorm = 54.6895, GNorm = 2.3060, lr_0 = 8.6198e-04
Validation auc = 0.708920
Epoch 95
Validation auc = 0.715023
Epoch 96
Validation auc = 0.704695
Epoch 97
Validation auc = 0.696714
Epoch 98
Validation auc = 0.682629
Epoch 99
Loss = 6.9108e-02, PNorm = 54.7686, GNorm = 1.4551, lr_0 = 8.5461e-04
Validation auc = 0.675587
Epoch 100
Validation auc = 0.697183
Epoch 101
Validation auc = 0.717840
Epoch 102
Validation auc = 0.728169
Epoch 103
Validation auc = 0.726291
Epoch 104
Loss = 1.2220e-02, PNorm = 54.8584, GNorm = 0.7824, lr_0 = 8.4730e-04
Validation auc = 0.726291
Epoch 105
Validation auc = 0.725352
Epoch 106
Validation auc = 0.716432
Epoch 107
Validation auc = 0.711737
Epoch 108
Validation auc = 0.708451
Epoch 109
Loss = 8.2807e-02, PNorm = 54.9353, GNorm = 1.4534, lr_0 = 8.4006e-04
Validation auc = 0.711737
Epoch 110
Validation auc = 0.719718
Epoch 111
Validation auc = 0.721596
Epoch 112
Validation auc = 0.721596
Epoch 113
Validation auc = 0.725352
Epoch 114
Loss = 7.2143e-03, PNorm = 55.0039, GNorm = 0.1023, lr_0 = 8.3288e-04
Validation auc = 0.724413
Epoch 115
Validation auc = 0.718310
Epoch 116
Validation auc = 0.709390
Epoch 117
Validation auc = 0.710798
Epoch 118
Validation auc = 0.716432
Epoch 119
Loss = 8.1794e-02, PNorm = 55.0855, GNorm = 0.2798, lr_0 = 8.2576e-04
Validation auc = 0.717840
Epoch 120
Validation auc = 0.717371
Epoch 121
Validation auc = 0.719249
Epoch 122
Validation auc = 0.721127
Epoch 123
Validation auc = 0.719718
Epoch 124
Loss = 5.7203e-02, PNorm = 55.1700, GNorm = 0.5325, lr_0 = 8.1870e-04
Validation auc = 0.718779
Epoch 125
Validation auc = 0.719249
Epoch 126
Validation auc = 0.715493
Epoch 127
Validation auc = 0.716901
Epoch 128
Validation auc = 0.707981
Epoch 129
Loss = 5.8980e-03, PNorm = 55.2449, GNorm = 0.3204, lr_0 = 8.1170e-04
Validation auc = 0.703286
Epoch 130
Validation auc = 0.710329
Epoch 131
Validation auc = 0.722535
Epoch 132
Validation auc = 0.725352
Epoch 133
Validation auc = 0.727700
Epoch 134
Loss = 6.2224e-03, PNorm = 55.3064, GNorm = 0.0593, lr_0 = 8.0476e-04
Validation auc = 0.727700
Epoch 135
Validation auc = 0.731455
Epoch 136
Validation auc = 0.730516
Epoch 137
Validation auc = 0.733803
Epoch 138
Validation auc = 0.728638
Epoch 139
Loss = 1.4136e-02, PNorm = 55.3658, GNorm = 0.5700, lr_0 = 7.9788e-04
Validation auc = 0.717371
Epoch 140
Validation auc = 0.718310
Epoch 141
Validation auc = 0.720657
Epoch 142
Validation auc = 0.720657
Epoch 143
Validation auc = 0.720188
Epoch 144
Loss = 8.9003e-03, PNorm = 55.4179, GNorm = 0.3682, lr_0 = 7.9106e-04
Validation auc = 0.715962
Epoch 145
Validation auc = 0.705634
Epoch 146
Validation auc = 0.702347
Epoch 147
Validation auc = 0.700469
Epoch 148
Validation auc = 0.704225
Epoch 149
Loss = 2.2807e-02, PNorm = 55.4680, GNorm = 0.6075, lr_0 = 7.8430e-04
Validation auc = 0.726761
Epoch 150
Validation auc = 0.735681
Epoch 151
Validation auc = 0.739906
Epoch 152
Validation auc = 0.735211
Epoch 153
Validation auc = 0.729577
Epoch 154
Loss = 1.4289e-03, PNorm = 55.5244, GNorm = 0.0202, lr_0 = 7.7759e-04
Validation auc = 0.722535
Epoch 155
Validation auc = 0.722535
Epoch 156
Validation auc = 0.724883
Epoch 157
Validation auc = 0.728638
Epoch 158
Validation auc = 0.731925
Epoch 159
Loss = 1.4898e-02, PNorm = 55.5780, GNorm = 1.2068, lr_0 = 7.7095e-04
Validation auc = 0.733333
Epoch 160
Validation auc = 0.731455
Epoch 161
Validation auc = 0.727230
Epoch 162
Validation auc = 0.725822
Epoch 163
Validation auc = 0.734272
Epoch 164
Loss = 4.1169e-02, PNorm = 55.6585, GNorm = 1.4599, lr_0 = 7.6436e-04
Validation auc = 0.739437
Epoch 165
Validation auc = 0.739906
Epoch 166
Validation auc = 0.742254
Epoch 167
Validation auc = 0.747887
Epoch 168
Validation auc = 0.748357
Epoch 169
Loss = 3.3047e-02, PNorm = 55.7273, GNorm = 0.4478, lr_0 = 7.5782e-04
Validation auc = 0.747418
Epoch 170
Validation auc = 0.743192
Epoch 171
Validation auc = 0.737559
Epoch 172
Validation auc = 0.733803
Epoch 173
Validation auc = 0.732394
Epoch 174
Loss = 2.0973e-03, PNorm = 55.7957, GNorm = 0.0401, lr_0 = 7.5134e-04
Validation auc = 0.730986
Epoch 175
Validation auc = 0.730047
Epoch 176
Validation auc = 0.726291
Epoch 177
Validation auc = 0.721596
Epoch 178
Validation auc = 0.722066
Epoch 179
Loss = 3.3175e-03, PNorm = 55.8482, GNorm = 0.0247, lr_0 = 7.4492e-04
Validation auc = 0.721127
Epoch 180
Validation auc = 0.717840
Epoch 181
Validation auc = 0.714085
Epoch 182
Validation auc = 0.708920
Epoch 183
Validation auc = 0.707512
Epoch 184
Loss = 3.4231e-03, PNorm = 55.8844, GNorm = 0.0615, lr_0 = 7.3855e-04
Validation auc = 0.708451
Epoch 185
Validation auc = 0.717840
Epoch 186
Validation auc = 0.725352
Epoch 187
Validation auc = 0.727700
Epoch 188
Validation auc = 0.729577
Epoch 189
Loss = 2.5748e-02, PNorm = 55.9196, GNorm = 0.7193, lr_0 = 7.3224e-04
Validation auc = 0.726761
Epoch 190
Validation auc = 0.723474
Epoch 191
Validation auc = 0.717371
Epoch 192
Validation auc = 0.711268
Epoch 193
Validation auc = 0.711268
Epoch 194
Loss = 1.6228e-03, PNorm = 55.9588, GNorm = 0.0470, lr_0 = 7.2598e-04
Validation auc = 0.711737
Epoch 195
Validation auc = 0.709390
Epoch 196
Validation auc = 0.708920
Epoch 197
Validation auc = 0.710798
Epoch 198
Validation auc = 0.711268
Epoch 199
Loss = 7.9946e-03, PNorm = 55.9798, GNorm = 0.1689, lr_0 = 7.1977e-04
Validation auc = 0.713615
Epoch 200
Validation auc = 0.714554
Epoch 201
Validation auc = 0.717371
Epoch 202
Validation auc = 0.710798
Epoch 203
Validation auc = 0.706103
Epoch 204
Loss = 3.6875e-03, PNorm = 56.0079, GNorm = 0.0730, lr_0 = 7.1362e-04
Validation auc = 0.702582
Epoch 205
Validation auc = 0.700235
Epoch 206
Validation auc = 0.696714
Epoch 207
Validation auc = 0.695775
Epoch 208
Validation auc = 0.696714
Epoch 209
Loss = 4.6111e-02, PNorm = 56.0415, GNorm = 1.8028, lr_0 = 7.0752e-04
Validation auc = 0.700000
Epoch 210
Validation auc = 0.708451
Epoch 211
Validation auc = 0.717840
Epoch 212
Validation auc = 0.724883
Epoch 213
Validation auc = 0.728169
Epoch 214
Loss = 9.3483e-02, PNorm = 56.0796, GNorm = 0.6128, lr_0 = 7.0147e-04
Validation auc = 0.717371
Epoch 215
Validation auc = 0.708451
Epoch 216
Validation auc = 0.707981
Epoch 217
Validation auc = 0.701878
Epoch 218
Validation auc = 0.704225
Epoch 219
Loss = 6.3637e-02, PNorm = 56.1252, GNorm = 2.2011, lr_0 = 6.9548e-04
Validation auc = 0.711737
Epoch 220
Validation auc = 0.725352
Epoch 221
Validation auc = 0.734272
Epoch 222
Validation auc = 0.733333
Epoch 223
Validation auc = 0.723709
Epoch 224
Loss = 2.0148e-02, PNorm = 56.1788, GNorm = 2.0871, lr_0 = 6.8953e-04
Validation auc = 0.712441
Epoch 225
Validation auc = 0.707981
Epoch 226
Validation auc = 0.707512
Epoch 227
Validation auc = 0.707981
Epoch 228
Validation auc = 0.707042
Epoch 229
Loss = 1.0217e-02, PNorm = 56.2289, GNorm = 0.8349, lr_0 = 6.8364e-04
Validation auc = 0.710329
Epoch 230
Validation auc = 0.710798
Epoch 231
Validation auc = 0.715023
Epoch 232
Validation auc = 0.716432
Epoch 233
Validation auc = 0.719718
Epoch 234
Loss = 4.0149e-03, PNorm = 56.2630, GNorm = 0.0984, lr_0 = 6.7779e-04
Validation auc = 0.719718
Epoch 235
Validation auc = 0.718310
Epoch 236
Validation auc = 0.714554
Epoch 237
Validation auc = 0.712207
Epoch 238
Validation auc = 0.708920
Epoch 239
Loss = 6.7850e-03, PNorm = 56.2852, GNorm = 0.6318, lr_0 = 6.7200e-04
Validation auc = 0.707512
Epoch 240
Validation auc = 0.717840
Epoch 241
Validation auc = 0.720188
Epoch 242
Validation auc = 0.722535
Epoch 243
Validation auc = 0.722066
Epoch 244
Loss = 1.7595e-02, PNorm = 56.3123, GNorm = 1.1748, lr_0 = 6.6625e-04
Validation auc = 0.720657
Epoch 245
Validation auc = 0.719249
Epoch 246
Validation auc = 0.718310
Epoch 247
Validation auc = 0.719718
Epoch 248
Validation auc = 0.723005
Epoch 249
Loss = 4.8815e-03, PNorm = 56.3389, GNorm = 0.0654, lr_0 = 6.6056e-04
Validation auc = 0.724883
Epoch 250
Validation auc = 0.723944
Epoch 251
Validation auc = 0.724883
Epoch 252
Validation auc = 0.724413
Epoch 253
Validation auc = 0.726291
Epoch 254
Loss = 4.3627e-04, PNorm = 56.3665, GNorm = 0.0332, lr_0 = 6.5491e-04
Validation auc = 0.724883
Epoch 255
Validation auc = 0.725822
Epoch 256
Validation auc = 0.724883
Epoch 257
Validation auc = 0.721127
Epoch 258
Validation auc = 0.720188
Epoch 259
Loss = 1.1651e-02, PNorm = 56.3972, GNorm = 0.0254, lr_0 = 6.4931e-04
Validation auc = 0.717371
Epoch 260
Validation auc = 0.718779
Epoch 261
Validation auc = 0.719953
Epoch 262
Validation auc = 0.719014
Epoch 263
Validation auc = 0.719014
Epoch 264
Loss = 1.1403e-03, PNorm = 56.4197, GNorm = 0.0309, lr_0 = 6.4376e-04
Validation auc = 0.719953
Epoch 265
Validation auc = 0.721596
Epoch 266
Validation auc = 0.723005
Epoch 267
Validation auc = 0.722535
Epoch 268
Validation auc = 0.723005
Epoch 269
Loss = 3.4726e-02, PNorm = 56.4366, GNorm = 1.1363, lr_0 = 6.3826e-04
Validation auc = 0.720188
Epoch 270
Validation auc = 0.719249
Epoch 271
Validation auc = 0.715962
Epoch 272
Validation auc = 0.713146
Epoch 273
Validation auc = 0.711268
Epoch 274
Loss = 2.1213e-03, PNorm = 56.4592, GNorm = 0.2297, lr_0 = 6.3280e-04
Validation auc = 0.710798
Epoch 275
Validation auc = 0.711268
Epoch 276
Validation auc = 0.713615
Epoch 277
Validation auc = 0.713380
Epoch 278
Validation auc = 0.714319
Epoch 279
Loss = 1.6166e-02, PNorm = 56.4813, GNorm = 1.0635, lr_0 = 6.2739e-04
Validation auc = 0.715493
Epoch 280
Validation auc = 0.718545
Epoch 281
Validation auc = 0.723944
Epoch 282
Validation auc = 0.727230
Epoch 283
Validation auc = 0.728638
Epoch 284
Loss = 5.0534e-02, PNorm = 56.5076, GNorm = 2.0849, lr_0 = 6.2203e-04
Validation auc = 0.730986
Epoch 285
Validation auc = 0.736620
Epoch 286
Validation auc = 0.740845
Epoch 287
Validation auc = 0.742723
Epoch 288
Validation auc = 0.744836
Epoch 289
Loss = 2.4736e-02, PNorm = 56.5353, GNorm = 1.4671, lr_0 = 6.1671e-04
Validation auc = 0.742254
Epoch 290
Validation auc = 0.739437
Epoch 291
Validation auc = 0.735211
Epoch 292
Validation auc = 0.734507
Epoch 293
Validation auc = 0.733803
Epoch 294
Loss = 6.4610e-02, PNorm = 56.5674, GNorm = 1.5109, lr_0 = 6.1144e-04
Validation auc = 0.732864
Epoch 295
Validation auc = 0.732394
Epoch 296
Validation auc = 0.729577
Epoch 297
Validation auc = 0.729108
Epoch 298
Validation auc = 0.726761
Epoch 299
Loss = 1.7600e-03, PNorm = 56.6180, GNorm = 0.1772, lr_0 = 6.0621e-04
Validation auc = 0.723474
Model 0 best validation auc = 0.762911 on epoch 10
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.927754
Ensemble test auc = 0.927754
Fold 6
Splitting data with seed 6
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation auc = 0.715023
Epoch 1
Validation auc = 0.721596
Epoch 2
Validation auc = 0.771831
Epoch 3
Validation auc = 0.782160
Epoch 4
Loss = 4.1211e-01, PNorm = 52.3070, GNorm = 2.1557, lr_0 = 6.5000e-04
Validation auc = 0.788263
Epoch 5
Validation auc = 0.792958
Epoch 6
Validation auc = 0.733333
Epoch 7
Validation auc = 0.751643
Epoch 8
Validation auc = 0.715962
Epoch 9
Loss = 2.0708e-01, PNorm = 52.5824, GNorm = 1.6487, lr_0 = 9.9743e-04
Validation auc = 0.747418
Epoch 10
Validation auc = 0.749765
Epoch 11
Validation auc = 0.745070
Epoch 12
Validation auc = 0.734742
Epoch 13
Validation auc = 0.725822
Epoch 14
Loss = 2.0340e-01, PNorm = 52.8861, GNorm = 1.9507, lr_0 = 9.8890e-04
Validation auc = 0.696244
Epoch 15
Validation auc = 0.700939
Epoch 16
Validation auc = 0.705164
Epoch 17
Validation auc = 0.721127
Epoch 18
Validation auc = 0.709859
Epoch 19
Loss = 1.5867e-01, PNorm = 53.1216, GNorm = 1.1947, lr_0 = 9.8045e-04
Validation auc = 0.714554
Epoch 20
Validation auc = 0.708451
Epoch 21
Validation auc = 0.723944
Epoch 22
Validation auc = 0.736150
Epoch 23
Validation auc = 0.750235
Epoch 24
Loss = 1.2686e-01, PNorm = 53.3245, GNorm = 4.7751, lr_0 = 9.7207e-04
Validation auc = 0.757746
Epoch 25
Validation auc = 0.740845
Epoch 26
Validation auc = 0.739906
Epoch 27
Validation auc = 0.754460
Epoch 28
Validation auc = 0.729108
Epoch 29
Loss = 6.7930e-02, PNorm = 53.5068, GNorm = 0.7297, lr_0 = 9.6376e-04
Validation auc = 0.711268
Epoch 30
Validation auc = 0.716432
Epoch 31
Validation auc = 0.726761
Epoch 32
Validation auc = 0.725352
Epoch 33
Validation auc = 0.728169
Epoch 34
Loss = 8.1502e-02, PNorm = 53.6707, GNorm = 1.4788, lr_0 = 9.5552e-04
Validation auc = 0.730516
Epoch 35
Validation auc = 0.746009
Epoch 36
Validation auc = 0.752113
Epoch 37
Validation auc = 0.747887
Epoch 38
Validation auc = 0.737089
Epoch 39
Loss = 1.3749e-01, PNorm = 53.8013, GNorm = 2.4359, lr_0 = 9.4735e-04
Validation auc = 0.745070
Epoch 40
Validation auc = 0.748357
Epoch 41
Validation auc = 0.723474
Epoch 42
Validation auc = 0.714554
Epoch 43
Validation auc = 0.714085
Epoch 44
Loss = 1.5788e-01, PNorm = 53.9183, GNorm = 2.5601, lr_0 = 9.3925e-04
Validation auc = 0.730516
Epoch 45
Validation auc = 0.736150
Epoch 46
Validation auc = 0.739437
Epoch 47
Validation auc = 0.749765
Epoch 48
Validation auc = 0.750704
Epoch 49
Loss = 6.7334e-02, PNorm = 54.0261, GNorm = 1.4423, lr_0 = 9.3122e-04
Validation auc = 0.744601
Epoch 50
Validation auc = 0.730516
Epoch 51
Validation auc = 0.715493
Epoch 52
Validation auc = 0.712676
Epoch 53
Validation auc = 0.719718
Epoch 54
Loss = 6.3859e-02, PNorm = 54.1289, GNorm = 0.3742, lr_0 = 9.2326e-04
Validation auc = 0.723005
Epoch 55
Validation auc = 0.724413
Epoch 56
Validation auc = 0.728638
Epoch 57
Validation auc = 0.726291
Epoch 58
Validation auc = 0.730047
Epoch 59
Loss = 9.1800e-02, PNorm = 54.2218, GNorm = 1.0846, lr_0 = 9.1537e-04
Validation auc = 0.730047
Epoch 60
Validation auc = 0.724883
Epoch 61
Validation auc = 0.728638
Epoch 62
Validation auc = 0.737089
Epoch 63
Validation auc = 0.735681
Epoch 64
Loss = 5.5347e-02, PNorm = 54.3129, GNorm = 1.9653, lr_0 = 9.0754e-04
Validation auc = 0.733803
Epoch 65
Validation auc = 0.732394
Epoch 66
Validation auc = 0.722535
Epoch 67
Validation auc = 0.714554
Epoch 68
Validation auc = 0.709390
Epoch 69
Loss = 1.5823e-01, PNorm = 54.3834, GNorm = 2.1950, lr_0 = 8.9978e-04
Validation auc = 0.714085
Epoch 70
Validation auc = 0.725352
Epoch 71
Validation auc = 0.727230
Epoch 72
Validation auc = 0.730047
Epoch 73
Validation auc = 0.723474
Epoch 74
Loss = 6.9845e-03, PNorm = 54.4650, GNorm = 0.2158, lr_0 = 8.9209e-04
Validation auc = 0.721127
Epoch 75
Validation auc = 0.723474
Epoch 76
Validation auc = 0.726291
Epoch 77
Validation auc = 0.727700
Epoch 78
Validation auc = 0.728638
Epoch 79
Loss = 5.5207e-03, PNorm = 54.5365, GNorm = 0.1216, lr_0 = 8.8447e-04
Validation auc = 0.725822
Epoch 80
Validation auc = 0.717840
Epoch 81
Validation auc = 0.716432
Epoch 82
Validation auc = 0.709859
Epoch 83
Validation auc = 0.706573
Epoch 84
Loss = 4.7403e-02, PNorm = 54.5969, GNorm = 0.3501, lr_0 = 8.7691e-04
Validation auc = 0.720657
Epoch 85
Validation auc = 0.729577
Epoch 86
Validation auc = 0.736150
Epoch 87
Validation auc = 0.736620
Epoch 88
Validation auc = 0.735211
Epoch 89
Loss = 3.2350e-02, PNorm = 54.6632, GNorm = 1.7446, lr_0 = 8.6941e-04
Validation auc = 0.736620
Epoch 90
Validation auc = 0.738028
Epoch 91
Validation auc = 0.737089
Epoch 92
Validation auc = 0.737089
Epoch 93
Validation auc = 0.739437
Epoch 94
Loss = 7.6153e-03, PNorm = 54.7131, GNorm = 0.3558, lr_0 = 8.6198e-04
Validation auc = 0.742254
Epoch 95
Validation auc = 0.743192
Epoch 96
Validation auc = 0.745540
Epoch 97
Validation auc = 0.744601
Epoch 98
Validation auc = 0.739437
Epoch 99
Loss = 3.2300e-03, PNorm = 54.7624, GNorm = 0.1697, lr_0 = 8.5461e-04
Validation auc = 0.733803
Epoch 100
Validation auc = 0.731455
Epoch 101
Validation auc = 0.737089
Epoch 102
Validation auc = 0.742723
Epoch 103
Validation auc = 0.736620
Epoch 104
Loss = 6.8166e-03, PNorm = 54.8131, GNorm = 0.3626, lr_0 = 8.4730e-04
Validation auc = 0.736620
Epoch 105
Validation auc = 0.735681
Epoch 106
Validation auc = 0.731455
Epoch 107
Validation auc = 0.735681
Epoch 108
Validation auc = 0.732394
Epoch 109
Loss = 6.2559e-03, PNorm = 54.8789, GNorm = 0.5855, lr_0 = 8.4006e-04
Validation auc = 0.730516
Epoch 110
Validation auc = 0.724883
Epoch 111
Validation auc = 0.714085
Epoch 112
Validation auc = 0.710798
Epoch 113
Validation auc = 0.707042
Epoch 114
Loss = 1.9643e-02, PNorm = 54.9475, GNorm = 0.8349, lr_0 = 8.3288e-04
Validation auc = 0.709859
Epoch 115
Validation auc = 0.722535
Epoch 116
Validation auc = 0.722066
Epoch 117
Validation auc = 0.728169
Epoch 118
Validation auc = 0.734272
Epoch 119
Loss = 2.3750e-02, PNorm = 55.0251, GNorm = 0.1828, lr_0 = 8.2576e-04
Validation auc = 0.737559
Epoch 120
Validation auc = 0.736150
Epoch 121
Validation auc = 0.730047
Epoch 122
Validation auc = 0.726761
Epoch 123
Validation auc = 0.724413
Epoch 124
Loss = 2.5717e-02, PNorm = 55.1150, GNorm = 0.6010, lr_0 = 8.1870e-04
Validation auc = 0.724883
Epoch 125
Validation auc = 0.723474
Epoch 126
Validation auc = 0.730047
Epoch 127
Validation auc = 0.730986
Epoch 128
Validation auc = 0.731925
Epoch 129
Loss = 4.2985e-02, PNorm = 55.2061, GNorm = 0.9082, lr_0 = 8.1170e-04
Validation auc = 0.731925
Epoch 130
Validation auc = 0.726291
Epoch 131
Validation auc = 0.729108
Epoch 132
Validation auc = 0.727700
Epoch 133
Validation auc = 0.729108
Epoch 134
Loss = 3.4169e-02, PNorm = 55.3348, GNorm = 0.7984, lr_0 = 8.0476e-04
Validation auc = 0.719249
Epoch 135
Validation auc = 0.721127
Epoch 136
Validation auc = 0.715962
Epoch 137
Validation auc = 0.721362
Epoch 138
Validation auc = 0.712676
Epoch 139
Loss = 3.1636e-02, PNorm = 55.4448, GNorm = 2.0830, lr_0 = 7.9788e-04
Validation auc = 0.715023
Epoch 140
Validation auc = 0.709859
Epoch 141
Validation auc = 0.710798
Epoch 142
Validation auc = 0.719718
Epoch 143
Validation auc = 0.734272
Epoch 144
Loss = 1.7891e-02, PNorm = 55.5288, GNorm = 1.1696, lr_0 = 7.9106e-04
Validation auc = 0.742254
Epoch 145
Validation auc = 0.748357
Epoch 146
Validation auc = 0.751643
Epoch 147
Validation auc = 0.754930
Epoch 148
Validation auc = 0.755399
Epoch 149
Loss = 6.2839e-02, PNorm = 55.5980, GNorm = 1.6653, lr_0 = 7.8430e-04
Validation auc = 0.754460
Epoch 150
Validation auc = 0.747418
Epoch 151
Validation auc = 0.742254
Epoch 152
Validation auc = 0.736620
Epoch 153
Validation auc = 0.732394
Epoch 154
Loss = 2.4776e-02, PNorm = 55.6396, GNorm = 0.1211, lr_0 = 7.7759e-04
Validation auc = 0.719718
Epoch 155
Validation auc = 0.713146
Epoch 156
Validation auc = 0.712676
Epoch 157
Validation auc = 0.715962
Epoch 158
Validation auc = 0.717840
Epoch 159
Loss = 2.5361e-03, PNorm = 55.6801, GNorm = 0.1443, lr_0 = 7.7095e-04
Validation auc = 0.721127
Epoch 160
Validation auc = 0.717371
Epoch 161
Validation auc = 0.716901
Epoch 162
Validation auc = 0.717371
Epoch 163
Validation auc = 0.724883
Epoch 164
Loss = 9.7613e-03, PNorm = 55.7204, GNorm = 0.4077, lr_0 = 7.6436e-04
Validation auc = 0.733803
Epoch 165
Validation auc = 0.743192
Epoch 166
Validation auc = 0.748826
Epoch 167
Validation auc = 0.749296
Epoch 168
Validation auc = 0.746948
Epoch 169
Loss = 1.5848e-02, PNorm = 55.7716, GNorm = 0.0351, lr_0 = 7.5782e-04
Validation auc = 0.740845
Epoch 170
Validation auc = 0.730516
Epoch 171
Validation auc = 0.722535
Epoch 172
Validation auc = 0.729577
Epoch 173
Validation auc = 0.728169
Epoch 174
Loss = 2.3706e-02, PNorm = 55.8295, GNorm = 0.3127, lr_0 = 7.5134e-04
Validation auc = 0.730516
Epoch 175
Validation auc = 0.731925
Epoch 176
Validation auc = 0.730516
Epoch 177
Validation auc = 0.731455
Epoch 178
Validation auc = 0.730516
Epoch 179
Loss = 2.6878e-03, PNorm = 55.8899, GNorm = 0.0462, lr_0 = 7.4492e-04
Validation auc = 0.728638
Epoch 180
Validation auc = 0.724883
Epoch 181
Validation auc = 0.723944
Epoch 182
Validation auc = 0.724413
Epoch 183
Validation auc = 0.722535
Epoch 184
Loss = 6.1038e-04, PNorm = 55.9310, GNorm = 0.0125, lr_0 = 7.3855e-04
Validation auc = 0.721127
Epoch 185
Validation auc = 0.720657
Epoch 186
Validation auc = 0.722535
Epoch 187
Validation auc = 0.722535
Epoch 188
Validation auc = 0.723944
Epoch 189
Loss = 9.8400e-03, PNorm = 55.9523, GNorm = 0.3749, lr_0 = 7.3224e-04
Validation auc = 0.721596
Epoch 190
Validation auc = 0.720188
Epoch 191
Validation auc = 0.718310
Epoch 192
Validation auc = 0.715023
Epoch 193
Validation auc = 0.710798
Epoch 194
Loss = 4.3369e-03, PNorm = 55.9858, GNorm = 0.0285, lr_0 = 7.2598e-04
Validation auc = 0.703756
Epoch 195
Validation auc = 0.703286
Epoch 196
Validation auc = 0.709390
Epoch 197
Validation auc = 0.709859
Epoch 198
Validation auc = 0.711737
Epoch 199
Loss = 2.3288e-02, PNorm = 55.9977, GNorm = 1.9676, lr_0 = 7.1977e-04
Validation auc = 0.712676
Epoch 200
Validation auc = 0.707512
Epoch 201
Validation auc = 0.709390
Epoch 202
Validation auc = 0.710798
Epoch 203
Validation auc = 0.717840
Epoch 204
Loss = 3.1356e-03, PNorm = 56.0122, GNorm = 0.0320, lr_0 = 7.1362e-04
Validation auc = 0.716901
Epoch 205
Validation auc = 0.716901
Epoch 206
Validation auc = 0.715493
Epoch 207
Validation auc = 0.710329
Epoch 208
Validation auc = 0.717371
Epoch 209
Loss = 2.7310e-02, PNorm = 56.0436, GNorm = 2.1936, lr_0 = 7.0752e-04
Validation auc = 0.727700
Epoch 210
Validation auc = 0.735681
Epoch 211
Validation auc = 0.741315
Epoch 212
Validation auc = 0.740376
Epoch 213
Validation auc = 0.736620
Epoch 214
Loss = 7.5596e-03, PNorm = 56.0861, GNorm = 0.8157, lr_0 = 7.0147e-04
Validation auc = 0.732160
Epoch 215
Validation auc = 0.729577
Epoch 216
Validation auc = 0.725822
Epoch 217
Validation auc = 0.729577
Epoch 218
Validation auc = 0.734742
Epoch 219
Loss = 1.2580e-02, PNorm = 56.1243, GNorm = 0.2442, lr_0 = 6.9548e-04
Validation auc = 0.736620
Epoch 220
Validation auc = 0.738028
Epoch 221
Validation auc = 0.738498
Epoch 222
Validation auc = 0.738967
Epoch 223
Validation auc = 0.740376
Epoch 224
Loss = 4.0012e-04, PNorm = 56.1538, GNorm = 0.0103, lr_0 = 6.8953e-04
Validation auc = 0.739437
Epoch 225
Validation auc = 0.740376
Epoch 226
Validation auc = 0.743662
Epoch 227
Validation auc = 0.744601
Epoch 228
Validation auc = 0.744601
Epoch 229
Loss = 3.6105e-02, PNorm = 56.1780, GNorm = 1.8263, lr_0 = 6.8364e-04
Validation auc = 0.743192
Epoch 230
Validation auc = 0.741784
Epoch 231
Validation auc = 0.744131
Epoch 232
Validation auc = 0.744601
Epoch 233
Validation auc = 0.742723
Epoch 234
Loss = 2.7710e-03, PNorm = 56.2141, GNorm = 0.1174, lr_0 = 6.7779e-04
Validation auc = 0.743192
Epoch 235
Validation auc = 0.741315
Epoch 236
Validation auc = 0.740376
Epoch 237
Validation auc = 0.737089
Epoch 238
Validation auc = 0.737089
Epoch 239
Loss = 6.8059e-03, PNorm = 56.2525, GNorm = 0.0300, lr_0 = 6.7200e-04
Validation auc = 0.736620
Epoch 240
Validation auc = 0.737089
Epoch 241
Validation auc = 0.734742
Epoch 242
Validation auc = 0.735681
Epoch 243
Validation auc = 0.732864
Epoch 244
Loss = 5.9099e-03, PNorm = 56.2826, GNorm = 0.4174, lr_0 = 6.6625e-04
Validation auc = 0.731455
Epoch 245
Validation auc = 0.727230
Epoch 246
Validation auc = 0.727700
Epoch 247
Validation auc = 0.725822
Epoch 248
Validation auc = 0.726761
Epoch 249
Loss = 4.1570e-02, PNorm = 56.3071, GNorm = 1.7096, lr_0 = 6.6056e-04
Validation auc = 0.730516
Epoch 250
Validation auc = 0.731455
Epoch 251
Validation auc = 0.732864
Epoch 252
Validation auc = 0.735211
Epoch 253
Validation auc = 0.736150
Epoch 254
Loss = 3.8568e-03, PNorm = 56.3272, GNorm = 0.5276, lr_0 = 6.5491e-04
Validation auc = 0.734272
Epoch 255
Validation auc = 0.732864
Epoch 256
Validation auc = 0.735211
Epoch 257
Validation auc = 0.731925
Epoch 258
Validation auc = 0.730047
Epoch 259
Loss = 5.2111e-04, PNorm = 56.3494, GNorm = 0.0350, lr_0 = 6.4931e-04
Validation auc = 0.728638
Epoch 260
Validation auc = 0.725822
Epoch 261
Validation auc = 0.726761
Epoch 262
Validation auc = 0.726291
Epoch 263
Validation auc = 0.727230
Epoch 264
Loss = 7.8351e-04, PNorm = 56.3746, GNorm = 0.0027, lr_0 = 6.4376e-04
Validation auc = 0.726526
Epoch 265
Validation auc = 0.728404
Epoch 266
Validation auc = 0.734507
Epoch 267
Validation auc = 0.736854
Epoch 268
Validation auc = 0.739202
Epoch 269
Loss = 1.8294e-03, PNorm = 56.4028, GNorm = 0.2763, lr_0 = 6.3826e-04
Validation auc = 0.734977
Epoch 270
Validation auc = 0.732160
Epoch 271
Validation auc = 0.731690
Epoch 272
Validation auc = 0.727465
Epoch 273
Validation auc = 0.726056
Epoch 274
Loss = 2.0457e-02, PNorm = 56.4238, GNorm = 1.6306, lr_0 = 6.3280e-04
Validation auc = 0.724178
Epoch 275
Validation auc = 0.725822
Epoch 276
Validation auc = 0.729577
Epoch 277
Validation auc = 0.730047
Epoch 278
Validation auc = 0.728638
Epoch 279
Loss = 2.5839e-02, PNorm = 56.4427, GNorm = 1.2100, lr_0 = 6.2739e-04
Validation auc = 0.729577
Epoch 280
Validation auc = 0.727230
Epoch 281
Validation auc = 0.728169
Epoch 282
Validation auc = 0.731455
Epoch 283
Validation auc = 0.731925
Epoch 284
Loss = 5.0122e-04, PNorm = 56.4729, GNorm = 0.0097, lr_0 = 6.2203e-04
Validation auc = 0.730986
Epoch 285
Validation auc = 0.729577
Epoch 286
Validation auc = 0.728638
Epoch 287
Validation auc = 0.730047
Epoch 288
Validation auc = 0.730516
Epoch 289
Loss = 9.0071e-04, PNorm = 56.4942, GNorm = 0.0271, lr_0 = 6.1671e-04
Validation auc = 0.732394
Epoch 290
Validation auc = 0.732629
Epoch 291
Validation auc = 0.732160
Epoch 292
Validation auc = 0.733099
Epoch 293
Validation auc = 0.732629
Epoch 294
Loss = 4.5306e-04, PNorm = 56.5113, GNorm = 0.0068, lr_0 = 6.1144e-04
Validation auc = 0.730751
Epoch 295
Validation auc = 0.730751
Epoch 296
Validation auc = 0.727700
Epoch 297
Validation auc = 0.726761
Epoch 298
Validation auc = 0.725352
Epoch 299
Loss = 2.1943e-04, PNorm = 56.5277, GNorm = 0.0149, lr_0 = 6.0621e-04
Validation auc = 0.724883
Model 0 best validation auc = 0.792958 on epoch 5
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.891632
Ensemble test auc = 0.891632
Fold 7
Splitting data with seed 7
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation auc = 0.705164
Epoch 1
Validation auc = 0.737089
Epoch 2
Validation auc = 0.736620
Epoch 3
Validation auc = 0.746479
Epoch 4
Loss = 6.6344e-01, PNorm = 52.3036, GNorm = 2.7656, lr_0 = 6.5000e-04
Validation auc = 0.714085
Epoch 5
Validation auc = 0.691549
Epoch 6
Validation auc = 0.727230
Epoch 7
Validation auc = 0.728638
Epoch 8
Validation auc = 0.737089
Epoch 9
Loss = 2.4664e-01, PNorm = 52.5613, GNorm = 2.6613, lr_0 = 9.9743e-04
Validation auc = 0.726761
Epoch 10
Validation auc = 0.748357
Epoch 11
Validation auc = 0.742254
Epoch 12
Validation auc = 0.725352
Epoch 13
Validation auc = 0.731925
Epoch 14
Loss = 1.5239e-01, PNorm = 52.8417, GNorm = 2.1312, lr_0 = 9.8890e-04
Validation auc = 0.730516
Epoch 15
Validation auc = 0.733803
Epoch 16
Validation auc = 0.740845
Epoch 17
Validation auc = 0.734742
Epoch 18
Validation auc = 0.726761
Epoch 19
Loss = 9.4473e-02, PNorm = 53.0635, GNorm = 1.1869, lr_0 = 9.8045e-04
Validation auc = 0.726291
Epoch 20
Validation auc = 0.730986
Epoch 21
Validation auc = 0.737089
Epoch 22
Validation auc = 0.750235
Epoch 23
Validation auc = 0.754460
Epoch 24
Loss = 6.0678e-02, PNorm = 53.2341, GNorm = 1.8498, lr_0 = 9.7207e-04
Validation auc = 0.744601
Epoch 25
Validation auc = 0.742254
Epoch 26
Validation auc = 0.756338
Epoch 27
Validation auc = 0.756808
Epoch 28
Validation auc = 0.729108
Epoch 29
Loss = 1.4796e-01, PNorm = 53.3636, GNorm = 2.6079, lr_0 = 9.6376e-04
Validation auc = 0.727230
Epoch 30
Validation auc = 0.717371
Epoch 31
Validation auc = 0.738498
Epoch 32
Validation auc = 0.764789
Epoch 33
Validation auc = 0.780751
Epoch 34
Loss = 9.4256e-02, PNorm = 53.4999, GNorm = 1.5290, lr_0 = 9.5552e-04
Validation auc = 0.784038
Epoch 35
Validation auc = 0.776526
Epoch 36
Validation auc = 0.763850
Epoch 37
Validation auc = 0.739906
Epoch 38
Validation auc = 0.722535
Epoch 39
Loss = 6.7373e-02, PNorm = 53.6253, GNorm = 1.6066, lr_0 = 9.4735e-04
Validation auc = 0.737089
Epoch 40
Validation auc = 0.736150
Epoch 41
Validation auc = 0.728169
Epoch 42
Validation auc = 0.730986
Epoch 43
Validation auc = 0.730986
Epoch 44
Loss = 1.3384e-01, PNorm = 53.7477, GNorm = 2.9904, lr_0 = 9.3925e-04
Validation auc = 0.733803
Epoch 45
Validation auc = 0.736150
Epoch 46
Validation auc = 0.733333
Epoch 47
Validation auc = 0.734742
Epoch 48
Validation auc = 0.733333
Epoch 49
Loss = 1.4773e-02, PNorm = 53.8562, GNorm = 0.2993, lr_0 = 9.3122e-04
Validation auc = 0.735211
Epoch 50
Validation auc = 0.740845
Epoch 51
Validation auc = 0.742254
Epoch 52
Validation auc = 0.732864
Epoch 53
Validation auc = 0.723474
Epoch 54
Loss = 4.6202e-02, PNorm = 53.9516, GNorm = 0.7187, lr_0 = 9.2326e-04
Validation auc = 0.715962
Epoch 55
Validation auc = 0.723944
Epoch 56
Validation auc = 0.734742
Epoch 57
Validation auc = 0.738967
Epoch 58
Validation auc = 0.742254
Epoch 59
Loss = 2.7451e-02, PNorm = 54.0554, GNorm = 0.7225, lr_0 = 9.1537e-04
Validation auc = 0.741315
Epoch 60
Validation auc = 0.733803
Epoch 61
Validation auc = 0.727700
Epoch 62
Validation auc = 0.722066
Epoch 63
Validation auc = 0.715493
Epoch 64
Loss = 5.5203e-02, PNorm = 54.1320, GNorm = 0.4968, lr_0 = 9.0754e-04
Validation auc = 0.720188
Epoch 65
Validation auc = 0.735681
Epoch 66
Validation auc = 0.745540
Epoch 67
Validation auc = 0.741315
Epoch 68
Validation auc = 0.730516
Epoch 69
Loss = 1.3107e-01, PNorm = 54.2021, GNorm = 4.3931, lr_0 = 8.9978e-04
Validation auc = 0.736150
Epoch 70
Validation auc = 0.760094
Epoch 71
Validation auc = 0.751174
Epoch 72
Validation auc = 0.748826
Epoch 73
Validation auc = 0.746479
Epoch 74
Loss = 2.0317e-01, PNorm = 54.3177, GNorm = 2.5068, lr_0 = 8.9209e-04
Validation auc = 0.756808
Epoch 75
Validation auc = 0.757746
Epoch 76
Validation auc = 0.746948
Epoch 77
Validation auc = 0.732864
Epoch 78
Validation auc = 0.724883
Epoch 79
Loss = 1.9343e-02, PNorm = 54.4450, GNorm = 0.4464, lr_0 = 8.8447e-04
Validation auc = 0.719249
Epoch 80
Validation auc = 0.715493
Epoch 81
Validation auc = 0.716901
Epoch 82
Validation auc = 0.718779
Epoch 83
Validation auc = 0.723005
Epoch 84
Loss = 3.0284e-02, PNorm = 54.5765, GNorm = 0.2857, lr_0 = 8.7691e-04
Validation auc = 0.718779
Epoch 85
Validation auc = 0.715962
Epoch 86
Validation auc = 0.719718
Epoch 87
Validation auc = 0.720188
Epoch 88
Validation auc = 0.722535
Epoch 89
Loss = 4.4585e-02, PNorm = 54.6681, GNorm = 0.4755, lr_0 = 8.6941e-04
Validation auc = 0.726761
Epoch 90
Validation auc = 0.727700
Epoch 91
Validation auc = 0.723005
Epoch 92
Validation auc = 0.722535
Epoch 93
Validation auc = 0.731925
Epoch 94
Loss = 1.9258e-02, PNorm = 54.7392, GNorm = 1.2331, lr_0 = 8.6198e-04
Validation auc = 0.746009
Epoch 95
Validation auc = 0.747887
Epoch 96
Validation auc = 0.744131
Epoch 97
Validation auc = 0.745070
Epoch 98
Validation auc = 0.743192
Epoch 99
Loss = 2.1442e-02, PNorm = 54.8066, GNorm = 1.3431, lr_0 = 8.5461e-04
Validation auc = 0.741784
Epoch 100
Validation auc = 0.746009
Epoch 101
Validation auc = 0.746948
Epoch 102
Validation auc = 0.749061
Epoch 103
Validation auc = 0.751643
Epoch 104
Loss = 6.9104e-03, PNorm = 54.8723, GNorm = 0.5670, lr_0 = 8.4730e-04
Validation auc = 0.754930
Epoch 105
Validation auc = 0.756338
Epoch 106
Validation auc = 0.751643
Epoch 107
Validation auc = 0.748357
Epoch 108
Validation auc = 0.743192
Epoch 109
Loss = 9.9515e-03, PNorm = 54.9254, GNorm = 0.4299, lr_0 = 8.4006e-04
Validation auc = 0.735681
Epoch 110
Validation auc = 0.730986
Epoch 111
Validation auc = 0.730986
Epoch 112
Validation auc = 0.729577
Epoch 113
Validation auc = 0.727700
Epoch 114
Loss = 1.0649e-01, PNorm = 54.9818, GNorm = 1.6843, lr_0 = 8.3288e-04
Validation auc = 0.731455
Epoch 115
Validation auc = 0.737089
Epoch 116
Validation auc = 0.740376
Epoch 117
Validation auc = 0.734272
Epoch 118
Validation auc = 0.726761
Epoch 119
Loss = 4.3028e-02, PNorm = 55.0311, GNorm = 0.3686, lr_0 = 8.2576e-04
Validation auc = 0.729108
Epoch 120
Validation auc = 0.725352
Epoch 121
Validation auc = 0.723474
Epoch 122
Validation auc = 0.721596
Epoch 123
Validation auc = 0.721127
Epoch 124
Loss = 1.7561e-02, PNorm = 55.0734, GNorm = 0.6806, lr_0 = 8.1870e-04
Validation auc = 0.720188
Epoch 125
Validation auc = 0.719718
Epoch 126
Validation auc = 0.722066
Epoch 127
Validation auc = 0.724883
Epoch 128
Validation auc = 0.726761
Epoch 129
Loss = 2.3113e-02, PNorm = 55.1115, GNorm = 0.0548, lr_0 = 8.1170e-04
Validation auc = 0.720188
Epoch 130
Validation auc = 0.716432
Epoch 131
Validation auc = 0.714554
Epoch 132
Validation auc = 0.715962
Epoch 133
Validation auc = 0.726291
Epoch 134
Loss = 2.1961e-03, PNorm = 55.1501, GNorm = 0.0445, lr_0 = 8.0476e-04
Validation auc = 0.732394
Epoch 135
Validation auc = 0.737089
Epoch 136
Validation auc = 0.739906
Epoch 137
Validation auc = 0.740845
Epoch 138
Validation auc = 0.728638
Epoch 139
Loss = 3.1435e-02, PNorm = 55.2066, GNorm = 2.1514, lr_0 = 7.9788e-04
Validation auc = 0.723005
Epoch 140
Validation auc = 0.731925
Epoch 141
Validation auc = 0.732864
Epoch 142
Validation auc = 0.726291
Epoch 143
Validation auc = 0.722535
Epoch 144
Loss = 3.7502e-02, PNorm = 55.2768, GNorm = 0.0430, lr_0 = 7.9106e-04
Validation auc = 0.718779
Epoch 145
Validation auc = 0.718779
Epoch 146
Validation auc = 0.715493
Epoch 147
Validation auc = 0.715493
Epoch 148
Validation auc = 0.719718
Epoch 149
Loss = 6.5835e-03, PNorm = 55.3349, GNorm = 0.1929, lr_0 = 7.8430e-04
Validation auc = 0.730986
Epoch 150
Validation auc = 0.727700
Epoch 151
Validation auc = 0.729577
Epoch 152
Validation auc = 0.730986
Epoch 153
Validation auc = 0.737089
Epoch 154
Loss = 9.5715e-03, PNorm = 55.3934, GNorm = 0.3355, lr_0 = 7.7759e-04
Validation auc = 0.739906
Epoch 155
Validation auc = 0.739906
Epoch 156
Validation auc = 0.738967
Epoch 157
Validation auc = 0.737089
Epoch 158
Validation auc = 0.736620
Epoch 159
Loss = 2.5456e-02, PNorm = 55.4329, GNorm = 2.2841, lr_0 = 7.7095e-04
Validation auc = 0.736620
Epoch 160
Validation auc = 0.743192
Epoch 161
Validation auc = 0.740376
Epoch 162
Validation auc = 0.732394
Epoch 163
Validation auc = 0.722066
Epoch 164
Loss = 1.8628e-02, PNorm = 55.4756, GNorm = 1.0516, lr_0 = 7.6436e-04
Validation auc = 0.719484
Epoch 165
Validation auc = 0.715962
Epoch 166
Validation auc = 0.719249
Epoch 167
Validation auc = 0.723474
Epoch 168
Validation auc = 0.732864
Epoch 169
Loss = 6.7085e-03, PNorm = 55.5287, GNorm = 0.3337, lr_0 = 7.5782e-04
Validation auc = 0.738028
Epoch 170
Validation auc = 0.740376
Epoch 171
Validation auc = 0.742254
Epoch 172
Validation auc = 0.742254
Epoch 173
Validation auc = 0.737559
Epoch 174
Loss = 5.4942e-04, PNorm = 55.5782, GNorm = 0.0131, lr_0 = 7.5134e-04
Validation auc = 0.734272
Epoch 175
Validation auc = 0.733803
Epoch 176
Validation auc = 0.729577
Epoch 177
Validation auc = 0.730047
Epoch 178
Validation auc = 0.730047
Epoch 179
Loss = 6.4683e-03, PNorm = 55.6061, GNorm = 0.0085, lr_0 = 7.4492e-04
Validation auc = 0.736620
Epoch 180
Validation auc = 0.738967
Epoch 181
Validation auc = 0.730986
Epoch 182
Validation auc = 0.718779
Epoch 183
Validation auc = 0.712207
Epoch 184
Loss = 5.8658e-03, PNorm = 55.6457, GNorm = 0.3789, lr_0 = 7.3855e-04
Validation auc = 0.704225
Epoch 185
Validation auc = 0.710329
Epoch 186
Validation auc = 0.725822
Epoch 187
Validation auc = 0.734742
Epoch 188
Validation auc = 0.749296
Epoch 189
Loss = 8.0703e-02, PNorm = 55.6940, GNorm = 1.9726, lr_0 = 7.3224e-04
Validation auc = 0.743662
Epoch 190
Validation auc = 0.736620
Epoch 191
Validation auc = 0.735211
Epoch 192
Validation auc = 0.733803
Epoch 193
Validation auc = 0.733803
Epoch 194
Loss = 4.6806e-02, PNorm = 55.7721, GNorm = 0.0833, lr_0 = 7.2598e-04
Validation auc = 0.736150
Epoch 195
Validation auc = 0.732864
Epoch 196
Validation auc = 0.728638
Epoch 197
Validation auc = 0.729108
Epoch 198
Validation auc = 0.723474
Epoch 199
Loss = 1.1726e-02, PNorm = 55.8333, GNorm = 0.3833, lr_0 = 7.1977e-04
Validation auc = 0.718310
Epoch 200
Validation auc = 0.718779
Epoch 201
Validation auc = 0.721127
Epoch 202
Validation auc = 0.722535
Epoch 203
Validation auc = 0.723005
Epoch 204
Loss = 1.0250e-02, PNorm = 55.8781, GNorm = 0.9461, lr_0 = 7.1362e-04
Validation auc = 0.723944
Epoch 205
Validation auc = 0.721127
Epoch 206
Validation auc = 0.722535
Epoch 207
Validation auc = 0.728169
Epoch 208
Validation auc = 0.735211
Epoch 209
Loss = 8.2031e-03, PNorm = 55.9263, GNorm = 0.1127, lr_0 = 7.0752e-04
Validation auc = 0.735681
Epoch 210
Validation auc = 0.734272
Epoch 211
Validation auc = 0.733803
Epoch 212
Validation auc = 0.731455
Epoch 213
Validation auc = 0.726761
Epoch 214
Loss = 8.3907e-03, PNorm = 55.9713, GNorm = 0.2714, lr_0 = 7.0147e-04
Validation auc = 0.727230
Epoch 215
Validation auc = 0.723944
Epoch 216
Validation auc = 0.721127
Epoch 217
Validation auc = 0.721596
Epoch 218
Validation auc = 0.721127
Epoch 219
Loss = 3.2559e-03, PNorm = 56.0033, GNorm = 0.1452, lr_0 = 6.9548e-04
Validation auc = 0.722066
Epoch 220
Validation auc = 0.722535
Epoch 221
Validation auc = 0.722535
Epoch 222
Validation auc = 0.722535
Epoch 223
Validation auc = 0.724883
Epoch 224
Loss = 6.2328e-03, PNorm = 56.0229, GNorm = 0.0078, lr_0 = 6.8953e-04
Validation auc = 0.725352
Epoch 225
Validation auc = 0.727700
Epoch 226
Validation auc = 0.728169
Epoch 227
Validation auc = 0.728169
Epoch 228
Validation auc = 0.729108
Epoch 229
Loss = 4.3816e-03, PNorm = 56.0390, GNorm = 0.4413, lr_0 = 6.8364e-04
Validation auc = 0.729108
Epoch 230
Validation auc = 0.728638
Epoch 231
Validation auc = 0.729577
Epoch 232
Validation auc = 0.730516
Epoch 233
Validation auc = 0.730986
Epoch 234
Loss = 3.7669e-03, PNorm = 56.0536, GNorm = 0.5463, lr_0 = 6.7779e-04
Validation auc = 0.729812
Epoch 235
Validation auc = 0.729577
Epoch 236
Validation auc = 0.730047
Epoch 237
Validation auc = 0.732394
Epoch 238
Validation auc = 0.731925
Epoch 239
Loss = 2.1909e-03, PNorm = 56.0724, GNorm = 0.2754, lr_0 = 6.7200e-04
Validation auc = 0.731455
Epoch 240
Validation auc = 0.731455
Epoch 241
Validation auc = 0.732394
Epoch 242
Validation auc = 0.734742
Epoch 243
Validation auc = 0.736150
Epoch 244
Loss = 1.2734e-03, PNorm = 56.0861, GNorm = 0.1332, lr_0 = 6.6625e-04
Validation auc = 0.736150
Epoch 245
Validation auc = 0.736620
Epoch 246
Validation auc = 0.733803
Epoch 247
Validation auc = 0.733803
Epoch 248
Validation auc = 0.731925
Epoch 249
Loss = 4.5012e-04, PNorm = 56.0982, GNorm = 0.0039, lr_0 = 6.6056e-04
Validation auc = 0.731455
Epoch 250
Validation auc = 0.732394
Epoch 251
Validation auc = 0.729577
Epoch 252
Validation auc = 0.729577
Epoch 253
Validation auc = 0.728169
Epoch 254
Loss = 2.9416e-03, PNorm = 56.1181, GNorm = 0.0411, lr_0 = 6.5491e-04
Validation auc = 0.732394
Epoch 255
Validation auc = 0.732864
Epoch 256
Validation auc = 0.734038
Epoch 257
Validation auc = 0.735681
Epoch 258
Validation auc = 0.738498
Epoch 259
Loss = 8.1570e-04, PNorm = 56.1367, GNorm = 0.0290, lr_0 = 6.4931e-04
Validation auc = 0.739437
Epoch 260
Validation auc = 0.740376
Epoch 261
Validation auc = 0.738967
Epoch 262
Validation auc = 0.736620
Epoch 263
Validation auc = 0.735681
Epoch 264
Loss = 1.5623e-03, PNorm = 56.1521, GNorm = 0.0739, lr_0 = 6.4376e-04
Validation auc = 0.734272
Epoch 265
Validation auc = 0.735211
Epoch 266
Validation auc = 0.735681
Epoch 267
Validation auc = 0.734742
Epoch 268
Validation auc = 0.734272
Epoch 269
Loss = 1.1254e-03, PNorm = 56.1718, GNorm = 0.0275, lr_0 = 6.3826e-04
Validation auc = 0.734742
Epoch 270
Validation auc = 0.736150
Epoch 271
Validation auc = 0.737559
Epoch 272
Validation auc = 0.738967
Epoch 273
Validation auc = 0.738028
Epoch 274
Loss = 1.4528e-03, PNorm = 56.1860, GNorm = 0.0917, lr_0 = 6.3280e-04
Validation auc = 0.736620
Epoch 275
Validation auc = 0.733099
Epoch 276
Validation auc = 0.730282
Epoch 277
Validation auc = 0.723005
Epoch 278
Validation auc = 0.718310
Epoch 279
Loss = 1.3314e-03, PNorm = 56.2024, GNorm = 0.1452, lr_0 = 6.2739e-04
Validation auc = 0.717136
Epoch 280
Validation auc = 0.713850
Epoch 281
Validation auc = 0.711737
Epoch 282
Validation auc = 0.711033
Epoch 283
Validation auc = 0.709155
Epoch 284
Loss = 4.3228e-04, PNorm = 56.2174, GNorm = 0.0044, lr_0 = 6.2203e-04
Validation auc = 0.708685
Epoch 285
Validation auc = 0.719718
Epoch 286
Validation auc = 0.722066
Epoch 287
Validation auc = 0.725822
Epoch 288
Validation auc = 0.727465
Epoch 289
Loss = 3.5627e-02, PNorm = 56.2309, GNorm = 0.2260, lr_0 = 6.1671e-04
Validation auc = 0.726291
Epoch 290
Validation auc = 0.726291
Epoch 291
Validation auc = 0.726995
Epoch 292
Validation auc = 0.726761
Epoch 293
Validation auc = 0.727700
Epoch 294
Loss = 3.4285e-04, PNorm = 56.2525, GNorm = 0.0059, lr_0 = 6.1144e-04
Validation auc = 0.725822
Epoch 295
Validation auc = 0.725352
Epoch 296
Validation auc = 0.725352
Epoch 297
Validation auc = 0.722066
Epoch 298
Validation auc = 0.723944
Epoch 299
Loss = 1.2508e-03, PNorm = 56.2733, GNorm = 0.1467, lr_0 = 6.0621e-04
Validation auc = 0.723944
Model 0 best validation auc = 0.784038 on epoch 34
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.904877
Ensemble test auc = 0.904877
Fold 8
Splitting data with seed 8
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation auc = 0.718779
Epoch 1
Validation auc = 0.726291
Epoch 2
Validation auc = 0.730986
Epoch 3
Validation auc = 0.753521
Epoch 4
Loss = 5.5947e-01, PNorm = 52.2996, GNorm = 1.5220, lr_0 = 6.5000e-04
Validation auc = 0.716432
Epoch 5
Validation auc = 0.753991
Epoch 6
Validation auc = 0.741784
Epoch 7
Validation auc = 0.744131
Epoch 8
Validation auc = 0.743662
Epoch 9
Loss = 2.2606e-01, PNorm = 52.5625, GNorm = 1.8941, lr_0 = 9.9743e-04
Validation auc = 0.735211
Epoch 10
Validation auc = 0.723474
Epoch 11
Validation auc = 0.717371
Epoch 12
Validation auc = 0.698592
Epoch 13
Validation auc = 0.718310
Epoch 14
Loss = 2.6995e-01, PNorm = 52.8601, GNorm = 4.8547, lr_0 = 9.8890e-04
Validation auc = 0.734272
Epoch 15
Validation auc = 0.732394
Epoch 16
Validation auc = 0.752113
Epoch 17
Validation auc = 0.774648
Epoch 18
Validation auc = 0.756338
Epoch 19
Loss = 1.0064e-01, PNorm = 53.1068, GNorm = 1.4896, lr_0 = 9.8045e-04
Validation auc = 0.727700
Epoch 20
Validation auc = 0.722066
Epoch 21
Validation auc = 0.721596
Epoch 22
Validation auc = 0.709859
Epoch 23
Validation auc = 0.707512
Epoch 24
Loss = 1.2704e-01, PNorm = 53.2938, GNorm = 1.1422, lr_0 = 9.7207e-04
Validation auc = 0.726761
Epoch 25
Validation auc = 0.731925
Epoch 26
Validation auc = 0.735211
Epoch 27
Validation auc = 0.728169
Epoch 28
Validation auc = 0.725822
Epoch 29
Loss = 1.0582e-01, PNorm = 53.4473, GNorm = 1.2507, lr_0 = 9.6376e-04
Validation auc = 0.728638
Epoch 30
Validation auc = 0.728638
Epoch 31
Validation auc = 0.710798
Epoch 32
Validation auc = 0.702347
Epoch 33
Validation auc = 0.716432
Epoch 34
Loss = 1.3079e-01, PNorm = 53.5810, GNorm = 1.0622, lr_0 = 9.5552e-04
Validation auc = 0.730047
Epoch 35
Validation auc = 0.730047
Epoch 36
Validation auc = 0.722066
Epoch 37
Validation auc = 0.723474
Epoch 38
Validation auc = 0.732864
Epoch 39
Loss = 4.3006e-02, PNorm = 53.7009, GNorm = 1.1082, lr_0 = 9.4735e-04
Validation auc = 0.731925
Epoch 40
Validation auc = 0.725352
Epoch 41
Validation auc = 0.714085
Epoch 42
Validation auc = 0.708451
Epoch 43
Validation auc = 0.717840
Epoch 44
Loss = 3.6298e-02, PNorm = 53.8118, GNorm = 0.5516, lr_0 = 9.3925e-04
Validation auc = 0.722535
Epoch 45
Validation auc = 0.721127
Epoch 46
Validation auc = 0.716901
Epoch 47
Validation auc = 0.716901
Epoch 48
Validation auc = 0.715493
Epoch 49
Loss = 2.9961e-02, PNorm = 53.9136, GNorm = 0.2770, lr_0 = 9.3122e-04
Validation auc = 0.714554
Epoch 50
Validation auc = 0.716901
Epoch 51
Validation auc = 0.715962
Epoch 52
Validation auc = 0.709390
Epoch 53
Validation auc = 0.712207
Epoch 54
Loss = 2.2134e-02, PNorm = 53.9947, GNorm = 1.1916, lr_0 = 9.2326e-04
Validation auc = 0.716901
Epoch 55
Validation auc = 0.720188
Epoch 56
Validation auc = 0.722066
Epoch 57
Validation auc = 0.720188
Epoch 58
Validation auc = 0.718310
Epoch 59
Loss = 8.8981e-02, PNorm = 54.0555, GNorm = 1.7146, lr_0 = 9.1537e-04
Validation auc = 0.723474
Epoch 60
Validation auc = 0.736150
Epoch 61
Validation auc = 0.736620
Epoch 62
Validation auc = 0.733803
Epoch 63
Validation auc = 0.729577
Epoch 64
Loss = 4.2627e-02, PNorm = 54.1416, GNorm = 1.4127, lr_0 = 9.0754e-04
Validation auc = 0.732394
Epoch 65
Validation auc = 0.730047
Epoch 66
Validation auc = 0.726761
Epoch 67
Validation auc = 0.711737
Epoch 68
Validation auc = 0.715493
Epoch 69
Loss = 1.3360e-01, PNorm = 54.2433, GNorm = 1.3130, lr_0 = 8.9978e-04
Validation auc = 0.724883
Epoch 70
Validation auc = 0.730751
Epoch 71
Validation auc = 0.731455
Epoch 72
Validation auc = 0.736620
Epoch 73
Validation auc = 0.738028
Epoch 74
Loss = 1.6238e-02, PNorm = 54.3356, GNorm = 0.4921, lr_0 = 8.9209e-04
Validation auc = 0.740376
Epoch 75
Validation auc = 0.738498
Epoch 76
Validation auc = 0.736620
Epoch 77
Validation auc = 0.734977
Epoch 78
Validation auc = 0.735211
Epoch 79
Loss = 2.6163e-02, PNorm = 54.4124, GNorm = 0.8041, lr_0 = 8.8447e-04
Validation auc = 0.735681
Epoch 80
Validation auc = 0.730516
Epoch 81
Validation auc = 0.725352
Epoch 82
Validation auc = 0.720657
Epoch 83
Validation auc = 0.730986
Epoch 84
Loss = 4.5750e-02, PNorm = 54.4831, GNorm = 1.7009, lr_0 = 8.7691e-04
Validation auc = 0.738498
Epoch 85
Validation auc = 0.731925
Epoch 86
Validation auc = 0.721596
Epoch 87
Validation auc = 0.708920
Epoch 88
Validation auc = 0.717371
Epoch 89
Loss = 2.1126e-02, PNorm = 54.5563, GNorm = 0.9111, lr_0 = 8.6941e-04
Validation auc = 0.724413
Epoch 90
Validation auc = 0.728638
Epoch 91
Validation auc = 0.736150
Epoch 92
Validation auc = 0.738498
Epoch 93
Validation auc = 0.735681
Epoch 94
Loss = 4.1739e-02, PNorm = 54.6306, GNorm = 1.5596, lr_0 = 8.6198e-04
Validation auc = 0.733803
Epoch 95
Validation auc = 0.729108
Epoch 96
Validation auc = 0.723239
Epoch 97
Validation auc = 0.722066
Epoch 98
Validation auc = 0.720188
Epoch 99
Loss = 3.0111e-02, PNorm = 54.7008, GNorm = 0.6671, lr_0 = 8.5461e-04
Validation auc = 0.720188
Epoch 100
Validation auc = 0.722066
Epoch 101
Validation auc = 0.730516
Epoch 102
Validation auc = 0.734742
Epoch 103
Validation auc = 0.733803
Epoch 104
Loss = 3.7841e-02, PNorm = 54.8055, GNorm = 1.2510, lr_0 = 8.4730e-04
Validation auc = 0.730516
Epoch 105
Validation auc = 0.738028
Epoch 106
Validation auc = 0.739437
Epoch 107
Validation auc = 0.739437
Epoch 108
Validation auc = 0.736150
Epoch 109
Loss = 9.3406e-02, PNorm = 54.9064, GNorm = 1.4648, lr_0 = 8.4006e-04
Validation auc = 0.740845
Epoch 110
Validation auc = 0.740376
Epoch 111
Validation auc = 0.750235
Epoch 112
Validation auc = 0.751643
Epoch 113
Validation auc = 0.756338
Epoch 114
Loss = 4.5062e-02, PNorm = 55.0250, GNorm = 0.8556, lr_0 = 8.3288e-04
Validation auc = 0.755399
Epoch 115
Validation auc = 0.751643
Epoch 116
Validation auc = 0.751643
Epoch 117
Validation auc = 0.751643
Epoch 118
Validation auc = 0.747418
Epoch 119
Loss = 2.6604e-02, PNorm = 55.1322, GNorm = 1.2469, lr_0 = 8.2576e-04
Validation auc = 0.736150
Epoch 120
Validation auc = 0.730516
Epoch 121
Validation auc = 0.735211
Epoch 122
Validation auc = 0.734742
Epoch 123
Validation auc = 0.740376
Epoch 124
Loss = 7.1891e-02, PNorm = 55.2355, GNorm = 2.7078, lr_0 = 8.1870e-04
Validation auc = 0.744131
Epoch 125
Validation auc = 0.748357
Epoch 126
Validation auc = 0.742723
Epoch 127
Validation auc = 0.738967
Epoch 128
Validation auc = 0.731925
Epoch 129
Loss = 8.8810e-02, PNorm = 55.3235, GNorm = 2.0161, lr_0 = 8.1170e-04
Validation auc = 0.726761
Epoch 130
Validation auc = 0.722066
Epoch 131
Validation auc = 0.722066
Epoch 132
Validation auc = 0.719249
Epoch 133
Validation auc = 0.719249
Epoch 134
Loss = 5.2374e-03, PNorm = 55.4012, GNorm = 0.3044, lr_0 = 8.0476e-04
Validation auc = 0.719249
Epoch 135
Validation auc = 0.717371
Epoch 136
Validation auc = 0.712676
Epoch 137
Validation auc = 0.710798
Epoch 138
Validation auc = 0.706103
Epoch 139
Loss = 2.8096e-02, PNorm = 55.4602, GNorm = 1.2619, lr_0 = 7.9788e-04
Validation auc = 0.707981
Epoch 140
Validation auc = 0.710798
Epoch 141
Validation auc = 0.713615
Epoch 142
Validation auc = 0.719249
Epoch 143
Validation auc = 0.719718
Epoch 144
Loss = 2.2045e-03, PNorm = 55.5164, GNorm = 0.0209, lr_0 = 7.9106e-04
Validation auc = 0.721127
Epoch 145
Validation auc = 0.727700
Epoch 146
Validation auc = 0.732394
Epoch 147
Validation auc = 0.730047
Epoch 148
Validation auc = 0.730516
Epoch 149
Loss = 1.5113e-03, PNorm = 55.5767, GNorm = 0.0244, lr_0 = 7.8430e-04
Validation auc = 0.731455
Epoch 150
Validation auc = 0.733099
Epoch 151
Validation auc = 0.725822
Epoch 152
Validation auc = 0.720188
Epoch 153
Validation auc = 0.717371
Epoch 154
Loss = 6.2156e-02, PNorm = 55.6204, GNorm = 1.5177, lr_0 = 7.7759e-04
Validation auc = 0.714085
Epoch 155
Validation auc = 0.721596
Epoch 156
Validation auc = 0.724413
Epoch 157
Validation auc = 0.719718
Epoch 158
Validation auc = 0.715023
Epoch 159
Loss = 1.5660e-02, PNorm = 55.6736, GNorm = 0.8919, lr_0 = 7.7095e-04
Validation auc = 0.714554
Epoch 160
Validation auc = 0.713380
Epoch 161
Validation auc = 0.713615
Epoch 162
Validation auc = 0.719249
Epoch 163
Validation auc = 0.726291
Epoch 164
Loss = 8.5637e-02, PNorm = 55.7252, GNorm = 0.7004, lr_0 = 7.6436e-04
Validation auc = 0.731925
Epoch 165
Validation auc = 0.734507
Epoch 166
Validation auc = 0.734742
Epoch 167
Validation auc = 0.734272
Epoch 168
Validation auc = 0.731925
Epoch 169
Loss = 3.0526e-02, PNorm = 55.7914, GNorm = 0.0166, lr_0 = 7.5782e-04
Validation auc = 0.729577
Epoch 170
Validation auc = 0.723005
Epoch 171
Validation auc = 0.719718
Epoch 172
Validation auc = 0.714085
Epoch 173
Validation auc = 0.713146
Epoch 174
Loss = 7.7292e-03, PNorm = 55.8618, GNorm = 0.6047, lr_0 = 7.5134e-04
Validation auc = 0.718779
Epoch 175
Validation auc = 0.717371
Epoch 176
Validation auc = 0.716901
Epoch 177
Validation auc = 0.714085
Epoch 178
Validation auc = 0.711737
Epoch 179
Loss = 5.9451e-03, PNorm = 55.9255, GNorm = 0.0562, lr_0 = 7.4492e-04
Validation auc = 0.707042
Epoch 180
Validation auc = 0.706103
Epoch 181
Validation auc = 0.706103
Epoch 182
Validation auc = 0.703756
Epoch 183
Validation auc = 0.703756
Epoch 184
Loss = 1.0193e-02, PNorm = 55.9668, GNorm = 0.4665, lr_0 = 7.3855e-04
Validation auc = 0.702347
Epoch 185
Validation auc = 0.705164
Epoch 186
Validation auc = 0.707042
Epoch 187
Validation auc = 0.707042
Epoch 188
Validation auc = 0.706573
Epoch 189
Loss = 4.0653e-02, PNorm = 55.9991, GNorm = 0.4564, lr_0 = 7.3224e-04
Validation auc = 0.702113
Epoch 190
Validation auc = 0.698592
Epoch 191
Validation auc = 0.696714
Epoch 192
Validation auc = 0.694836
Epoch 193
Validation auc = 0.696714
Epoch 194
Loss = 1.5428e-02, PNorm = 56.0455, GNorm = 1.2389, lr_0 = 7.2598e-04
Validation auc = 0.698592
Epoch 195
Validation auc = 0.700000
Epoch 196
Validation auc = 0.699531
Epoch 197
Validation auc = 0.697653
Epoch 198
Validation auc = 0.698592
Epoch 199
Loss = 3.2222e-02, PNorm = 56.0994, GNorm = 1.2655, lr_0 = 7.1977e-04
Validation auc = 0.700000
Epoch 200
Validation auc = 0.702817
Epoch 201
Validation auc = 0.705634
Epoch 202
Validation auc = 0.706338
Epoch 203
Validation auc = 0.704460
Epoch 204
Loss = 2.5227e-03, PNorm = 56.1394, GNorm = 0.0135, lr_0 = 7.1362e-04
Validation auc = 0.704225
Epoch 205
Validation auc = 0.701878
Epoch 206
Validation auc = 0.700000
Epoch 207
Validation auc = 0.700939
Epoch 208
Validation auc = 0.705634
Epoch 209
Loss = 5.1015e-03, PNorm = 56.1721, GNorm = 0.1298, lr_0 = 7.0752e-04
Validation auc = 0.702817
Epoch 210
Validation auc = 0.702817
Epoch 211
Validation auc = 0.698122
Epoch 212
Validation auc = 0.695775
Epoch 213
Validation auc = 0.697183
Epoch 214
Loss = 1.7465e-02, PNorm = 56.2086, GNorm = 0.4183, lr_0 = 7.0147e-04
Validation auc = 0.696244
Epoch 215
Validation auc = 0.700939
Epoch 216
Validation auc = 0.703286
Epoch 217
Validation auc = 0.703286
Epoch 218
Validation auc = 0.705634
Epoch 219
Loss = 9.8581e-03, PNorm = 56.2373, GNorm = 0.7912, lr_0 = 6.9548e-04
Validation auc = 0.708451
Epoch 220
Validation auc = 0.709859
Epoch 221
Validation auc = 0.707981
Epoch 222
Validation auc = 0.706103
Epoch 223
Validation auc = 0.700000
Epoch 224
Loss = 3.5271e-02, PNorm = 56.2678, GNorm = 0.0024, lr_0 = 6.8953e-04
Validation auc = 0.698122
Epoch 225
Validation auc = 0.697653
Epoch 226
Validation auc = 0.696244
Epoch 227
Validation auc = 0.700469
Epoch 228
Validation auc = 0.704695
Epoch 229
Loss = 1.4466e-02, PNorm = 56.3018, GNorm = 0.9298, lr_0 = 6.8364e-04
Validation auc = 0.709390
Epoch 230
Validation auc = 0.713146
Epoch 231
Validation auc = 0.715962
Epoch 232
Validation auc = 0.715493
Epoch 233
Validation auc = 0.712676
Epoch 234
Loss = 1.5086e-03, PNorm = 56.3390, GNorm = 0.0910, lr_0 = 6.7779e-04
Validation auc = 0.713615
Epoch 235
Validation auc = 0.707512
Epoch 236
Validation auc = 0.705164
Epoch 237
Validation auc = 0.706573
Epoch 238
Validation auc = 0.704695
Epoch 239
Loss = 1.6095e-03, PNorm = 56.3665, GNorm = 0.0363, lr_0 = 6.7200e-04
Validation auc = 0.706573
Epoch 240
Validation auc = 0.708451
Epoch 241
Validation auc = 0.711268
Epoch 242
Validation auc = 0.715962
Epoch 243
Validation auc = 0.717371
Epoch 244
Loss = 5.5922e-04, PNorm = 56.3922, GNorm = 0.0576, lr_0 = 6.6625e-04
Validation auc = 0.716901
Epoch 245
Validation auc = 0.715962
Epoch 246
Validation auc = 0.717371
Epoch 247
Validation auc = 0.717371
Epoch 248
Validation auc = 0.715962
Epoch 249
Loss = 1.1846e-03, PNorm = 56.4096, GNorm = 0.0079, lr_0 = 6.6056e-04
Validation auc = 0.714554
Epoch 250
Validation auc = 0.715023
Epoch 251
Validation auc = 0.717371
Epoch 252
Validation auc = 0.717840
Epoch 253
Validation auc = 0.719718
Epoch 254
Loss = 1.9131e-03, PNorm = 56.4301, GNorm = 0.0271, lr_0 = 6.5491e-04
Validation auc = 0.717840
Epoch 255
Validation auc = 0.719249
Epoch 256
Validation auc = 0.718779
Epoch 257
Validation auc = 0.718779
Epoch 258
Validation auc = 0.717840
Epoch 259
Loss = 7.2563e-03, PNorm = 56.4539, GNorm = 0.5356, lr_0 = 6.4931e-04
Validation auc = 0.718779
Epoch 260
Validation auc = 0.714085
Epoch 261
Validation auc = 0.715023
Epoch 262
Validation auc = 0.711268
Epoch 263
Validation auc = 0.706573
Epoch 264
Loss = 8.2713e-03, PNorm = 56.4839, GNorm = 1.0422, lr_0 = 6.4376e-04
Validation auc = 0.704225
Epoch 265
Validation auc = 0.706103
Epoch 266
Validation auc = 0.709859
Epoch 267
Validation auc = 0.710329
Epoch 268
Validation auc = 0.711268
Epoch 269
Loss = 1.2043e-02, PNorm = 56.5167, GNorm = 1.5365, lr_0 = 6.3826e-04
Validation auc = 0.711268
Epoch 270
Validation auc = 0.711268
Epoch 271
Validation auc = 0.712676
Epoch 272
Validation auc = 0.709859
Epoch 273
Validation auc = 0.706573
Epoch 274
Loss = 1.5328e-03, PNorm = 56.5550, GNorm = 0.0389, lr_0 = 6.3280e-04
Validation auc = 0.704695
Epoch 275
Validation auc = 0.702113
Epoch 276
Validation auc = 0.699296
Epoch 277
Validation auc = 0.698122
Epoch 278
Validation auc = 0.693427
Epoch 279
Loss = 2.3262e-03, PNorm = 56.5829, GNorm = 0.1173, lr_0 = 6.2739e-04
Validation auc = 0.692488
Epoch 280
Validation auc = 0.689671
Epoch 281
Validation auc = 0.690610
Epoch 282
Validation auc = 0.692254
Epoch 283
Validation auc = 0.694836
Epoch 284
Loss = 3.0482e-02, PNorm = 56.6043, GNorm = 1.9628, lr_0 = 6.2203e-04
Validation auc = 0.696244
Epoch 285
Validation auc = 0.699531
Epoch 286
Validation auc = 0.701408
Epoch 287
Validation auc = 0.706103
Epoch 288
Validation auc = 0.707981
Epoch 289
Loss = 3.6839e-04, PNorm = 56.6249, GNorm = 0.0335, lr_0 = 6.1671e-04
Validation auc = 0.708920
Epoch 290
Validation auc = 0.710329
Epoch 291
Validation auc = 0.711737
Epoch 292
Validation auc = 0.704695
Epoch 293
Validation auc = 0.700000
Epoch 294
Loss = 4.0359e-03, PNorm = 56.6491, GNorm = 0.0150, lr_0 = 6.1144e-04
Validation auc = 0.698122
Epoch 295
Validation auc = 0.697653
Epoch 296
Validation auc = 0.702347
Epoch 297
Validation auc = 0.704695
Epoch 298
Validation auc = 0.708451
Epoch 299
Loss = 8.3341e-04, PNorm = 56.6678, GNorm = 0.0562, lr_0 = 6.0621e-04
Validation auc = 0.712207
Model 0 best validation auc = 0.774648 on epoch 17
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.927152
Ensemble test auc = 0.927152
Fold 9
Splitting data with seed 9
Class sizes
sars_cov_two_cl_protease_active 0: 89.69%, 1: 10.31%
Total size = 485 | train size = 485 | val size = 157 | test size = 162
With class_balance, effective train size = 100
Building model 0
MoleculeModel(
  (sigmoid): Sigmoid()
  (encoder): MPN()
  (ffn): Sequential(
    (0): Dropout(p=0.1, inplace=False)
    (1): Linear(in_features=300, out_features=2200, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.1, inplace=False)
    (4): Linear(in_features=2200, out_features=2200, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.1, inplace=False)
    (7): Linear(in_features=2200, out_features=1, bias=True)
  )
)
Number of parameters = 5,506,601
Moving model to cuda
Epoch 0
Validation auc = 0.744601
Epoch 1
Validation auc = 0.764319
Epoch 2
Validation auc = 0.753521
Epoch 3
Validation auc = 0.767606
Epoch 4
Loss = 3.9416e-01, PNorm = 52.2987, GNorm = 2.7707, lr_0 = 6.5000e-04
Validation auc = 0.764319
Epoch 5
Validation auc = 0.761972
Epoch 6
Validation auc = 0.753991
Epoch 7
Validation auc = 0.753052
Epoch 8
Validation auc = 0.753052
Epoch 9
Loss = 2.6424e-01, PNorm = 52.5573, GNorm = 4.9716, lr_0 = 9.9743e-04
Validation auc = 0.745540
Epoch 10
Validation auc = 0.758685
Epoch 11
Validation auc = 0.744131
Epoch 12
Validation auc = 0.729108
Epoch 13
Validation auc = 0.715493
Epoch 14
Loss = 1.8249e-01, PNorm = 52.8597, GNorm = 1.9232, lr_0 = 9.8890e-04
Validation auc = 0.706573
Epoch 15
Validation auc = 0.723474
Epoch 16
Validation auc = 0.728638
Epoch 17
Validation auc = 0.725352
Epoch 18
Validation auc = 0.733333
Epoch 19
Loss = 1.7320e-01, PNorm = 53.1051, GNorm = 2.3017, lr_0 = 9.8045e-04
Validation auc = 0.742254
Epoch 20
Validation auc = 0.759155
Epoch 21
Validation auc = 0.771362
Epoch 22
Validation auc = 0.777465
Epoch 23
Validation auc = 0.784038
Epoch 24
Loss = 6.1022e-02, PNorm = 53.3035, GNorm = 1.5466, lr_0 = 9.7207e-04
Validation auc = 0.784038
Epoch 25
Validation auc = 0.775587
Epoch 26
Validation auc = 0.766667
Epoch 27
Validation auc = 0.752582
Epoch 28
Validation auc = 0.744601
Epoch 29
Loss = 7.1670e-02, PNorm = 53.4578, GNorm = 1.5272, lr_0 = 9.6376e-04
Validation auc = 0.719249
Epoch 30
Validation auc = 0.723005
Epoch 31
Validation auc = 0.723005
Epoch 32
Validation auc = 0.731455
Epoch 33
Validation auc = 0.743192
Epoch 34
Loss = 4.3714e-02, PNorm = 53.5859, GNorm = 0.6468, lr_0 = 9.5552e-04
Validation auc = 0.739906
Epoch 35
Validation auc = 0.754460
Epoch 36
Validation auc = 0.753991
Epoch 37
Validation auc = 0.746009
Epoch 38
Validation auc = 0.733803
Epoch 39
Loss = 1.8543e-01, PNorm = 53.6990, GNorm = 1.3627, lr_0 = 9.4735e-04
Validation auc = 0.717840
Epoch 40
Validation auc = 0.701408
Epoch 41
Validation auc = 0.697653
Epoch 42
Validation auc = 0.706103
Epoch 43
Validation auc = 0.715493
Epoch 44
Loss = 2.0226e-01, PNorm = 53.8289, GNorm = 2.7897, lr_0 = 9.3925e-04
Validation auc = 0.733803
Epoch 45
Validation auc = 0.731455
Epoch 46
Validation auc = 0.720657
Epoch 47
Validation auc = 0.756338
Epoch 48
Validation auc = 0.771362
Epoch 49
Loss = 1.2841e-01, PNorm = 53.9876, GNorm = 1.1598, lr_0 = 9.3122e-04
Validation auc = 0.765728
Epoch 50
Validation auc = 0.749296
Epoch 51
Validation auc = 0.736150
Epoch 52
Validation auc = 0.732394
Epoch 53
Validation auc = 0.748826
Epoch 54
Loss = 4.4511e-02, PNorm = 54.1357, GNorm = 0.9285, lr_0 = 9.2326e-04
Validation auc = 0.745540
Epoch 55
Validation auc = 0.741315
Epoch 56
Validation auc = 0.735681
Epoch 57
Validation auc = 0.730047
Epoch 58
Validation auc = 0.721127
Epoch 59
Loss = 8.0297e-02, PNorm = 54.2390, GNorm = 1.6912, lr_0 = 9.1537e-04
Validation auc = 0.719249
Epoch 60
Validation auc = 0.722066
Epoch 61
Validation auc = 0.734742
Epoch 62
Validation auc = 0.742723
Epoch 63
Validation auc = 0.745540
Epoch 64
Loss = 4.2324e-02, PNorm = 54.3245, GNorm = 0.6610, lr_0 = 9.0754e-04
Validation auc = 0.746009
Epoch 65
Validation auc = 0.745070
Epoch 66
Validation auc = 0.738967
Epoch 67
Validation auc = 0.725822
Epoch 68
Validation auc = 0.718779
Epoch 69
Loss = 7.4509e-02, PNorm = 54.4038, GNorm = 1.4258, lr_0 = 8.9978e-04
Validation auc = 0.726761
Epoch 70
Validation auc = 0.750704
Epoch 71
Validation auc = 0.764319
Epoch 72
Validation auc = 0.757746
Epoch 73
Validation auc = 0.761033
Epoch 74
Loss = 4.3409e-02, PNorm = 54.4954, GNorm = 1.3223, lr_0 = 8.9209e-04
Validation auc = 0.755869
Epoch 75
Validation auc = 0.756338
Epoch 76
Validation auc = 0.752582
Epoch 77
Validation auc = 0.747418
Epoch 78
Validation auc = 0.741784
Epoch 79
Loss = 8.0679e-02, PNorm = 54.5726, GNorm = 1.4223, lr_0 = 8.8447e-04
Validation auc = 0.739437
Epoch 80
Validation auc = 0.736620
Epoch 81
Validation auc = 0.738498
Epoch 82
Validation auc = 0.742723
Epoch 83
Validation auc = 0.743192
Epoch 84
Loss = 3.5270e-02, PNorm = 54.6444, GNorm = 2.1802, lr_0 = 8.7691e-04
Validation auc = 0.738967
Epoch 85
Validation auc = 0.734742
Epoch 86
Validation auc = 0.727230
Epoch 87
Validation auc = 0.729577
Epoch 88
Validation auc = 0.737089
Epoch 89
Loss = 5.1670e-02, PNorm = 54.7086, GNorm = 0.8632, lr_0 = 8.6941e-04
Validation auc = 0.745070
Epoch 90
Validation auc = 0.744131
Epoch 91
Validation auc = 0.741784
Epoch 92
Validation auc = 0.730516
Epoch 93
Validation auc = 0.721127
Epoch 94
Loss = 9.8768e-03, PNorm = 54.7714, GNorm = 0.6161, lr_0 = 8.6198e-04
Validation auc = 0.715962
Epoch 95
Validation auc = 0.721127
Epoch 96
Validation auc = 0.717840
Epoch 97
Validation auc = 0.702347
Epoch 98
Validation auc = 0.706103
Epoch 99
Loss = 6.2922e-02, PNorm = 54.8330, GNorm = 2.5193, lr_0 = 8.5461e-04
Validation auc = 0.734742
Epoch 100
Validation auc = 0.746009
Epoch 101
Validation auc = 0.758216
Epoch 102
Validation auc = 0.758216
Epoch 103
Validation auc = 0.743192
Epoch 104
Loss = 2.1641e-02, PNorm = 54.9229, GNorm = 0.2985, lr_0 = 8.4730e-04
Validation auc = 0.730047
Epoch 105
Validation auc = 0.723944
Epoch 106
Validation auc = 0.721127
Epoch 107
Validation auc = 0.718779
Epoch 108
Validation auc = 0.715962
Epoch 109
Loss = 2.9281e-02, PNorm = 54.9894, GNorm = 1.1378, lr_0 = 8.4006e-04
Validation auc = 0.715023
Epoch 110
Validation auc = 0.715728
Epoch 111
Validation auc = 0.718779
Epoch 112
Validation auc = 0.721596
Epoch 113
Validation auc = 0.722535
Epoch 114
Loss = 5.1008e-03, PNorm = 55.0384, GNorm = 0.2221, lr_0 = 8.3288e-04
Validation auc = 0.723474
Epoch 115
Validation auc = 0.724413
Epoch 116
Validation auc = 0.725822
Epoch 117
Validation auc = 0.726291
Epoch 118
Validation auc = 0.729108
Epoch 119
Loss = 2.6247e-02, PNorm = 55.0767, GNorm = 0.0596, lr_0 = 8.2576e-04
Validation auc = 0.725822
Epoch 120
Validation auc = 0.723944
Epoch 121
Validation auc = 0.723474
Epoch 122
Validation auc = 0.724413
Epoch 123
Validation auc = 0.722770
Epoch 124
Loss = 2.8191e-03, PNorm = 55.1094, GNorm = 0.0565, lr_0 = 8.1870e-04
Validation auc = 0.723474
Epoch 125
Validation auc = 0.724413
Epoch 126
Validation auc = 0.725822
Epoch 127
Validation auc = 0.723474
Epoch 128
Validation auc = 0.718779
Epoch 129
Loss = 9.6427e-03, PNorm = 55.1389, GNorm = 0.0278, lr_0 = 8.1170e-04
Validation auc = 0.715493
Epoch 130
Validation auc = 0.718310
Epoch 131
Validation auc = 0.720657
Epoch 132
Validation auc = 0.725352
Epoch 133
Validation auc = 0.726995
Epoch 134
Loss = 5.5017e-02, PNorm = 55.1716, GNorm = 1.2796, lr_0 = 8.0476e-04
Validation auc = 0.720657
Epoch 135
Validation auc = 0.716901
Epoch 136
Validation auc = 0.714085
Epoch 137
Validation auc = 0.714085
Epoch 138
Validation auc = 0.716432
Epoch 139
Loss = 2.6558e-02, PNorm = 55.2027, GNorm = 2.1291, lr_0 = 7.9788e-04
Validation auc = 0.716901
Epoch 140
Validation auc = 0.719718
Epoch 141
Validation auc = 0.718310
Epoch 142
Validation auc = 0.717371
Epoch 143
Validation auc = 0.718075
Epoch 144
Loss = 1.2415e-02, PNorm = 55.2354, GNorm = 0.2875, lr_0 = 7.9106e-04
Validation auc = 0.718779
Epoch 145
Validation auc = 0.720423
Epoch 146
Validation auc = 0.726291
Epoch 147
Validation auc = 0.719718
Epoch 148
Validation auc = 0.715258
Epoch 149
Loss = 6.3777e-02, PNorm = 55.2717, GNorm = 0.9981, lr_0 = 7.8430e-04
Validation auc = 0.719249
Epoch 150
Validation auc = 0.720423
Epoch 151
Validation auc = 0.720657
Epoch 152
Validation auc = 0.722535
Epoch 153
Validation auc = 0.719953
Epoch 154
Loss = 2.9794e-02, PNorm = 55.3185, GNorm = 1.9044, lr_0 = 7.7759e-04
Validation auc = 0.725352
Epoch 155
Validation auc = 0.731925
Epoch 156
Validation auc = 0.734272
Epoch 157
Validation auc = 0.735681
Epoch 158
Validation auc = 0.728638
Epoch 159
Loss = 1.8223e-02, PNorm = 55.3667, GNorm = 1.5446, lr_0 = 7.7095e-04
Validation auc = 0.721127
Epoch 160
Validation auc = 0.714789
Epoch 161
Validation auc = 0.719953
Epoch 162
Validation auc = 0.730516
Epoch 163
Validation auc = 0.732864
Epoch 164
Loss = 1.3857e-01, PNorm = 55.4572, GNorm = 0.1690, lr_0 = 7.6436e-04
Validation auc = 0.728169
Epoch 165
Validation auc = 0.724413
Epoch 166
Validation auc = 0.719249
Epoch 167
Validation auc = 0.722066
Epoch 168
Validation auc = 0.727230
Epoch 169
Loss = 2.4345e-02, PNorm = 55.5632, GNorm = 0.5311, lr_0 = 7.5782e-04
Validation auc = 0.734272
Epoch 170
Validation auc = 0.736620
Epoch 171
Validation auc = 0.731455
Epoch 172
Validation auc = 0.726291
Epoch 173
Validation auc = 0.716432
Epoch 174
Loss = 9.9562e-02, PNorm = 55.6619, GNorm = 1.6157, lr_0 = 7.5134e-04
Validation auc = 0.703286
Epoch 175
Validation auc = 0.708920
Epoch 176
Validation auc = 0.715493
Epoch 177
Validation auc = 0.712676
Epoch 178
Validation auc = 0.712207
Epoch 179
Loss = 1.6481e-02, PNorm = 55.7584, GNorm = 0.3635, lr_0 = 7.4492e-04
Validation auc = 0.710798
Epoch 180
Validation auc = 0.712207
Epoch 181
Validation auc = 0.715023
Epoch 182
Validation auc = 0.721127
Epoch 183
Validation auc = 0.725587
Epoch 184
Loss = 8.1796e-03, PNorm = 55.8300, GNorm = 0.4264, lr_0 = 7.3855e-04
Validation auc = 0.727230
Epoch 185
Validation auc = 0.728169
Epoch 186
Validation auc = 0.729108
Epoch 187
Validation auc = 0.730986
Epoch 188
Validation auc = 0.729577
Epoch 189
Loss = 2.7082e-03, PNorm = 55.8866, GNorm = 0.0574, lr_0 = 7.3224e-04
Validation auc = 0.726995
Epoch 190
Validation auc = 0.731455
Epoch 191
Validation auc = 0.730047
Epoch 192
Validation auc = 0.727230
Epoch 193
Validation auc = 0.731925
Epoch 194
Loss = 5.7928e-03, PNorm = 55.9324, GNorm = 0.2395, lr_0 = 7.2598e-04
Validation auc = 0.734742
Epoch 195
Validation auc = 0.738967
Epoch 196
Validation auc = 0.740845
Epoch 197
Validation auc = 0.740845
Epoch 198
Validation auc = 0.740845
Epoch 199
Loss = 3.2747e-03, PNorm = 55.9782, GNorm = 0.0821, lr_0 = 7.1977e-04
Validation auc = 0.740376
Epoch 200
Validation auc = 0.739906
Epoch 201
Validation auc = 0.737089
Epoch 202
Validation auc = 0.735211
Epoch 203
Validation auc = 0.733803
Epoch 204
Loss = 6.0085e-03, PNorm = 56.0064, GNorm = 0.0267, lr_0 = 7.1362e-04
Validation auc = 0.728638
Epoch 205
Validation auc = 0.724413
Epoch 206
Validation auc = 0.717840
Epoch 207
Validation auc = 0.722535
Epoch 208
Validation auc = 0.728169
Epoch 209
Loss = 2.1673e-03, PNorm = 56.0412, GNorm = 0.1459, lr_0 = 7.0752e-04
Validation auc = 0.731455
Epoch 210
Validation auc = 0.736150
Epoch 211
Validation auc = 0.737559
Epoch 212
Validation auc = 0.738498
Epoch 213
Validation auc = 0.738498
Epoch 214
Loss = 1.6499e-02, PNorm = 56.0775, GNorm = 0.1373, lr_0 = 7.0147e-04
Validation auc = 0.737559
Epoch 215
Validation auc = 0.734272
Epoch 216
Validation auc = 0.731455
Epoch 217
Validation auc = 0.730986
Epoch 218
Validation auc = 0.731455
Epoch 219
Loss = 1.0596e-03, PNorm = 56.1146, GNorm = 0.0316, lr_0 = 6.9548e-04
Validation auc = 0.729577
Epoch 220
Validation auc = 0.729108
Epoch 221
Validation auc = 0.729108
Epoch 222
Validation auc = 0.725822
Epoch 223
Validation auc = 0.724883
Epoch 224
Loss = 1.6240e-02, PNorm = 56.1432, GNorm = 0.0341, lr_0 = 6.8953e-04
Validation auc = 0.730047
Epoch 225
Validation auc = 0.731925
Epoch 226
Validation auc = 0.732394
Epoch 227
Validation auc = 0.735681
Epoch 228
Validation auc = 0.733803
Epoch 229
Loss = 7.8417e-03, PNorm = 56.1738, GNorm = 0.3643, lr_0 = 6.8364e-04
Validation auc = 0.732394
Epoch 230
Validation auc = 0.727700
Epoch 231
Validation auc = 0.725822
Epoch 232
Validation auc = 0.719249
Epoch 233
Validation auc = 0.708920
Epoch 234
Loss = 6.4605e-03, PNorm = 56.2069, GNorm = 0.0311, lr_0 = 6.7779e-04
Validation auc = 0.702817
Epoch 235
Validation auc = 0.699531
Epoch 236
Validation auc = 0.699531
Epoch 237
Validation auc = 0.699531
Epoch 238
Validation auc = 0.704695
Epoch 239
Loss = 1.1279e-02, PNorm = 56.2379, GNorm = 0.0538, lr_0 = 6.7200e-04
Validation auc = 0.715493
Epoch 240
Validation auc = 0.724413
Epoch 241
Validation auc = 0.729577
Epoch 242
Validation auc = 0.732394
Epoch 243
Validation auc = 0.736150
Epoch 244
Loss = 1.0111e-03, PNorm = 56.2655, GNorm = 0.0393, lr_0 = 6.6625e-04
Validation auc = 0.738028
Epoch 245
Validation auc = 0.737089
Epoch 246
Validation auc = 0.738028
Epoch 247
Validation auc = 0.738498
Epoch 248
Validation auc = 0.737089
Epoch 249
Loss = 3.5363e-03, PNorm = 56.2857, GNorm = 0.4676, lr_0 = 6.6056e-04
Validation auc = 0.734742
Epoch 250
Validation auc = 0.734272
Epoch 251
Validation auc = 0.731925
Epoch 252
Validation auc = 0.731455
Epoch 253
Validation auc = 0.730986
Epoch 254
Loss = 6.7317e-04, PNorm = 56.2996, GNorm = 0.0107, lr_0 = 6.5491e-04
Validation auc = 0.728638
Epoch 255
Validation auc = 0.728638
Epoch 256
Validation auc = 0.727230
Epoch 257
Validation auc = 0.727230
Epoch 258
Validation auc = 0.726761
Epoch 259
Loss = 2.9634e-04, PNorm = 56.3075, GNorm = 0.0040, lr_0 = 6.4931e-04
Validation auc = 0.727700
Epoch 260
Validation auc = 0.728638
Epoch 261
Validation auc = 0.730047
Epoch 262
Validation auc = 0.730516
Epoch 263
Validation auc = 0.730986
Epoch 264
Loss = 5.2295e-04, PNorm = 56.3138, GNorm = 0.0375, lr_0 = 6.4376e-04
Validation auc = 0.732394
Epoch 265
Validation auc = 0.732864
Epoch 266
Validation auc = 0.733803
Epoch 267
Validation auc = 0.733803
Epoch 268
Validation auc = 0.734272
Epoch 269
Loss = 2.3009e-03, PNorm = 56.3271, GNorm = 0.2834, lr_0 = 6.3826e-04
Validation auc = 0.731925
Epoch 270
Validation auc = 0.730986
Epoch 271
Validation auc = 0.727700
Epoch 272
Validation auc = 0.726761
Epoch 273
Validation auc = 0.726761
Epoch 274
Loss = 2.9950e-03, PNorm = 56.3433, GNorm = 0.2371, lr_0 = 6.3280e-04
Validation auc = 0.727230
Epoch 275
Validation auc = 0.727230
Epoch 276
Validation auc = 0.726761
Epoch 277
Validation auc = 0.727230
Epoch 278
Validation auc = 0.727700
Epoch 279
Loss = 4.7325e-03, PNorm = 56.3579, GNorm = 0.3911, lr_0 = 6.2739e-04
Validation auc = 0.727700
Epoch 280
Validation auc = 0.729108
Epoch 281
Validation auc = 0.728638
Epoch 282
Validation auc = 0.727230
Epoch 283
Validation auc = 0.727230
Epoch 284
Loss = 7.2256e-03, PNorm = 56.3758, GNorm = 0.9778, lr_0 = 6.2203e-04
Validation auc = 0.728169
Epoch 285
Validation auc = 0.728169
Epoch 286
Validation auc = 0.727700
Epoch 287
Validation auc = 0.727230
Epoch 288
Validation auc = 0.729108
Epoch 289
Loss = 1.2009e-03, PNorm = 56.3933, GNorm = 0.0792, lr_0 = 6.1671e-04
Validation auc = 0.728169
Epoch 290
Validation auc = 0.726761
Epoch 291
Validation auc = 0.726291
Epoch 292
Validation auc = 0.725822
Epoch 293
Validation auc = 0.725822
Epoch 294
Loss = 2.9871e-04, PNorm = 56.4099, GNorm = 0.0062, lr_0 = 6.1144e-04
Validation auc = 0.725822
Epoch 295
Validation auc = 0.725587
Epoch 296
Validation auc = 0.725587
Epoch 297
Validation auc = 0.727230
Epoch 298
Validation auc = 0.727700
Epoch 299
Loss = 1.7947e-03, PNorm = 56.4220, GNorm = 0.2340, lr_0 = 6.0621e-04
Validation auc = 0.729577
Model 0 best validation auc = 0.784038 on epoch 23
Loading pretrained parameter "ffn.1.weight".
Loading pretrained parameter "ffn.1.bias".
Loading pretrained parameter "ffn.4.weight".
Loading pretrained parameter "ffn.4.bias".
Loading pretrained parameter "ffn.7.weight".
Loading pretrained parameter "ffn.7.bias".
Moving model to cuda
Model 0 test auc = 0.881999
Ensemble test auc = 0.881999
10-fold cross validation
	Seed 0 ==> test auc = 0.941601
	Seed 1 ==> test auc = 0.886213
	Seed 2 ==> test auc = 0.894040
	Seed 3 ==> test auc = 0.791692
	Seed 4 ==> test auc = 0.931367
	Seed 5 ==> test auc = 0.927754
	Seed 6 ==> test auc = 0.891632
	Seed 7 ==> test auc = 0.904877
	Seed 8 ==> test auc = 0.927152
	Seed 9 ==> test auc = 0.881999
Overall test auc = 0.897833 +/- 0.040681
Elapsed time = 0:04:02
